2023-11-19 23:53:13,525 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-11-19 23:53:13,779 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b 2023-11-19 23:53:13,799 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-11-19 23:53:13,799 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d, deleteOnExit=true 2023-11-19 23:53:13,800 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-11-19 23:53:13,801 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/test.cache.data in system properties and HBase conf 2023-11-19 23:53:13,801 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/hadoop.tmp.dir in system properties and HBase conf 2023-11-19 23:53:13,801 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/hadoop.log.dir in system properties and HBase conf 2023-11-19 23:53:13,802 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-11-19 23:53:13,802 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-11-19 23:53:13,803 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-11-19 23:53:13,928 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-11-19 23:53:14,353 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-11-19 23:53:14,359 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-11-19 23:53:14,360 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-11-19 23:53:14,360 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-11-19 23:53:14,361 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-19 23:53:14,361 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-11-19 23:53:14,362 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-11-19 23:53:14,362 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-19 23:53:14,362 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-19 23:53:14,363 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-11-19 23:53:14,363 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/nfs.dump.dir in system properties and HBase conf 2023-11-19 23:53:14,364 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/java.io.tmpdir in system properties and HBase conf 2023-11-19 23:53:14,364 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-19 23:53:14,365 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-11-19 23:53:14,365 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-11-19 23:53:15,107 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-19 23:53:15,111 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-19 23:53:15,427 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-11-19 23:53:15,624 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-11-19 23:53:15,639 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-19 23:53:15,676 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-11-19 23:53:15,711 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/java.io.tmpdir/Jetty_localhost_37423_hdfs____.8mxl1h/webapp 2023-11-19 23:53:15,866 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37423 2023-11-19 23:53:15,877 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-19 23:53:15,877 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-19 23:53:16,406 WARN [Listener at localhost/45125] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-19 23:53:16,500 WARN [Listener at localhost/45125] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-19 23:53:16,524 WARN [Listener at localhost/45125] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-19 23:53:16,532 INFO [Listener at localhost/45125] log.Slf4jLog(67): jetty-6.1.26 2023-11-19 23:53:16,537 INFO [Listener at localhost/45125] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/java.io.tmpdir/Jetty_localhost_35225_datanode____sgbg6y/webapp 2023-11-19 23:53:16,637 INFO [Listener at localhost/45125] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35225 2023-11-19 23:53:17,046 WARN [Listener at localhost/36279] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-19 23:53:17,077 WARN [Listener at localhost/36279] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-19 23:53:17,088 WARN [Listener at localhost/36279] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-19 23:53:17,090 INFO [Listener at localhost/36279] log.Slf4jLog(67): jetty-6.1.26 2023-11-19 23:53:17,103 INFO [Listener at localhost/36279] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/java.io.tmpdir/Jetty_localhost_40105_datanode____dt5srl/webapp 2023-11-19 23:53:17,253 INFO [Listener at localhost/36279] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40105 2023-11-19 23:53:17,272 WARN [Listener at localhost/41985] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-19 23:53:17,319 WARN [Listener at localhost/41985] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-19 23:53:17,323 WARN [Listener at localhost/41985] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-19 23:53:17,327 INFO [Listener at localhost/41985] log.Slf4jLog(67): jetty-6.1.26 2023-11-19 23:53:17,336 INFO [Listener at localhost/41985] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/java.io.tmpdir/Jetty_localhost_33759_datanode____.gjxhpa/webapp 2023-11-19 23:53:17,487 INFO [Listener at localhost/41985] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33759 2023-11-19 23:53:17,535 WARN [Listener at localhost/42385] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-19 23:53:17,725 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed8e2df016e1aec4: Processing first storage report for DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b from datanode 92965f72-b1d2-4a0b-81fa-7e4974137a5a 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed8e2df016e1aec4: from storage DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b node DatanodeRegistration(127.0.0.1:36193, datanodeUuid=92965f72-b1d2-4a0b-81fa-7e4974137a5a, infoPort=46237, infoSecurePort=0, ipcPort=41985, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x47d67b961cc39878: Processing first storage report for DS-2897e77b-c5c0-4723-a296-ada3a640a3dd from datanode c1e6d43a-1c05-42be-a588-44ff607a74aa 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x47d67b961cc39878: from storage DS-2897e77b-c5c0-4723-a296-ada3a640a3dd node DatanodeRegistration(127.0.0.1:42551, datanodeUuid=c1e6d43a-1c05-42be-a588-44ff607a74aa, infoPort=40259, infoSecurePort=0, ipcPort=36279, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf74c69f369d49cfe: Processing first storage report for DS-066cee74-6470-4910-b337-571812d566b1 from datanode 3d39986b-89f7-4439-b67d-be51253160b9 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf74c69f369d49cfe: from storage DS-066cee74-6470-4910-b337-571812d566b1 node DatanodeRegistration(127.0.0.1:44679, datanodeUuid=3d39986b-89f7-4439-b67d-be51253160b9, infoPort=43853, infoSecurePort=0, ipcPort=42385, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xed8e2df016e1aec4: Processing first storage report for DS-4113f49f-4598-44f8-9f7c-ab00bfd46e84 from datanode 92965f72-b1d2-4a0b-81fa-7e4974137a5a 2023-11-19 23:53:17,727 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xed8e2df016e1aec4: from storage DS-4113f49f-4598-44f8-9f7c-ab00bfd46e84 node DatanodeRegistration(127.0.0.1:36193, datanodeUuid=92965f72-b1d2-4a0b-81fa-7e4974137a5a, infoPort=46237, infoSecurePort=0, ipcPort=41985, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,728 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x47d67b961cc39878: Processing first storage report for DS-95bca419-b44b-4eb3-8d76-acb1ca481763 from datanode c1e6d43a-1c05-42be-a588-44ff607a74aa 2023-11-19 23:53:17,728 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x47d67b961cc39878: from storage DS-95bca419-b44b-4eb3-8d76-acb1ca481763 node DatanodeRegistration(127.0.0.1:42551, datanodeUuid=c1e6d43a-1c05-42be-a588-44ff607a74aa, infoPort=40259, infoSecurePort=0, ipcPort=36279, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,728 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf74c69f369d49cfe: Processing first storage report for DS-e7adf98f-1893-42f5-adfa-255efcfdcd08 from datanode 3d39986b-89f7-4439-b67d-be51253160b9 2023-11-19 23:53:17,728 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf74c69f369d49cfe: from storage DS-e7adf98f-1893-42f5-adfa-255efcfdcd08 node DatanodeRegistration(127.0.0.1:44679, datanodeUuid=3d39986b-89f7-4439-b67d-be51253160b9, infoPort=43853, infoSecurePort=0, ipcPort=42385, storageInfo=lv=-57;cid=testClusterID;nsid=557685405;c=1700437995196), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-19 23:53:17,988 DEBUG [Listener at localhost/42385] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b 2023-11-19 23:53:18,134 INFO [Listener at localhost/42385] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/zookeeper_0, clientPort=52135, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-11-19 23:53:18,149 INFO [Listener at localhost/42385] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52135 2023-11-19 23:53:18,157 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:18,160 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:18,838 INFO [Listener at localhost/42385] util.FSUtils(471): Created version file at hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6 with version=8 2023-11-19 23:53:18,838 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/hbase-staging 2023-11-19 23:53:19,163 INFO [Listener at localhost/42385] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-11-19 23:53:19,722 INFO [Listener at localhost/42385] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-11-19 23:53:19,754 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:19,754 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:19,755 INFO [Listener at localhost/42385] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-19 23:53:19,755 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:19,755 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-19 23:53:19,911 INFO [Listener at localhost/42385] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-11-19 23:53:19,993 DEBUG [Listener at localhost/42385] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-11-19 23:53:20,097 INFO [Listener at localhost/42385] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45383 2023-11-19 23:53:20,108 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,110 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,138 INFO [Listener at localhost/42385] zookeeper.RecoverableZooKeeper(93): Process identifier=master:45383 connecting to ZooKeeper ensemble=127.0.0.1:52135 2023-11-19 23:53:20,184 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:453830x0, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-19 23:53:20,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:45383-0x10008109e390000 connected 2023-11-19 23:53:20,223 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-19 23:53:20,224 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:53:20,227 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-19 23:53:20,236 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45383 2023-11-19 23:53:20,236 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45383 2023-11-19 23:53:20,237 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45383 2023-11-19 23:53:20,238 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45383 2023-11-19 23:53:20,239 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45383 2023-11-19 23:53:20,245 INFO [Listener at localhost/42385] master.HMaster(444): hbase.rootdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6, hbase.cluster.distributed=false 2023-11-19 23:53:20,320 INFO [Listener at localhost/42385] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-11-19 23:53:20,320 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,321 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,321 INFO [Listener at localhost/42385] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-19 23:53:20,321 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,321 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-19 23:53:20,326 INFO [Listener at localhost/42385] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-19 23:53:20,328 INFO [Listener at localhost/42385] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36283 2023-11-19 23:53:20,331 INFO [Listener at localhost/42385] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-19 23:53:20,337 DEBUG [Listener at localhost/42385] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-19 23:53:20,339 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,341 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,344 INFO [Listener at localhost/42385] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36283 connecting to ZooKeeper ensemble=127.0.0.1:52135 2023-11-19 23:53:20,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:362830x0, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-19 23:53:20,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36283-0x10008109e390001 connected 2023-11-19 23:53:20,351 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-19 23:53:20,354 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:53:20,355 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-19 23:53:20,355 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36283 2023-11-19 23:53:20,356 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36283 2023-11-19 23:53:20,356 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36283 2023-11-19 23:53:20,357 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36283 2023-11-19 23:53:20,357 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36283 2023-11-19 23:53:20,374 INFO [Listener at localhost/42385] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-11-19 23:53:20,375 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,375 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,375 INFO [Listener at localhost/42385] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-19 23:53:20,376 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,376 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-19 23:53:20,376 INFO [Listener at localhost/42385] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-19 23:53:20,378 INFO [Listener at localhost/42385] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34057 2023-11-19 23:53:20,379 INFO [Listener at localhost/42385] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-19 23:53:20,380 DEBUG [Listener at localhost/42385] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-19 23:53:20,381 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,383 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,386 INFO [Listener at localhost/42385] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34057 connecting to ZooKeeper ensemble=127.0.0.1:52135 2023-11-19 23:53:20,391 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:340570x0, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-19 23:53:20,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34057-0x10008109e390002 connected 2023-11-19 23:53:20,392 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-19 23:53:20,393 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:53:20,394 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-19 23:53:20,395 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34057 2023-11-19 23:53:20,396 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34057 2023-11-19 23:53:20,397 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34057 2023-11-19 23:53:20,398 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34057 2023-11-19 23:53:20,400 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34057 2023-11-19 23:53:20,412 INFO [Listener at localhost/42385] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-11-19 23:53:20,412 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,412 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,412 INFO [Listener at localhost/42385] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-19 23:53:20,413 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-19 23:53:20,413 INFO [Listener at localhost/42385] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-19 23:53:20,413 INFO [Listener at localhost/42385] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-19 23:53:20,445 INFO [Listener at localhost/42385] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39613 2023-11-19 23:53:20,446 INFO [Listener at localhost/42385] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-19 23:53:20,447 DEBUG [Listener at localhost/42385] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-19 23:53:20,448 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,450 INFO [Listener at localhost/42385] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,452 INFO [Listener at localhost/42385] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39613 connecting to ZooKeeper ensemble=127.0.0.1:52135 2023-11-19 23:53:20,457 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:396130x0, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-19 23:53:20,458 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:396130x0, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-19 23:53:20,459 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:396130x0, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:53:20,460 DEBUG [Listener at localhost/42385] zookeeper.ZKUtil(164): regionserver:396130x0, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-19 23:53:20,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39613-0x10008109e390003 connected 2023-11-19 23:53:20,461 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39613 2023-11-19 23:53:20,462 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39613 2023-11-19 23:53:20,464 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39613 2023-11-19 23:53:20,464 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39613 2023-11-19 23:53:20,468 DEBUG [Listener at localhost/42385] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39613 2023-11-19 23:53:20,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:20,479 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-19 23:53:20,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:20,502 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-19 23:53:20,502 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-19 23:53:20,502 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-19 23:53:20,502 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-19 23:53:20,503 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:20,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-19 23:53:20,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,45383,1700437998977 from backup master directory 2023-11-19 23:53:20,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-19 23:53:20,512 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:20,512 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-19 23:53:20,513 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-19 23:53:20,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:20,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-11-19 23:53:20,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-11-19 23:53:20,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/hbase.id with ID: 2701879b-bdc6-444e-ba28-c316ec4514dc 2023-11-19 23:53:20,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-19 23:53:20,681 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:20,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2ca9643d to 127.0.0.1:52135 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-19 23:53:20,762 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79f4e4c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-19 23:53:20,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-19 23:53:20,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-11-19 23:53:20,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-11-19 23:53:20,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-11-19 23:53:20,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:53:20,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:53:20,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-19 23:53:20,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store-tmp 2023-11-19 23:53:20,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-19 23:53:20,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:53:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:53:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-19 23:53:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:53:20,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:53:20,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-19 23:53:20,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/WALs/jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:20,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45383%2C1700437998977, suffix=, logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/WALs/jenkins-hbase4.apache.org,45383,1700437998977, archiveDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/oldWALs, maxLogs=10 2023-11-19 23:53:21,018 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK] 2023-11-19 23:53:21,018 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK] 2023-11-19 23:53:21,018 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK] 2023-11-19 23:53:21,031 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:53:21,110 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/WALs/jenkins-hbase4.apache.org,45383,1700437998977/jenkins-hbase4.apache.org%2C45383%2C1700437998977.1700438000951 2023-11-19 23:53:21,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK], DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK]] 2023-11-19 23:53:21,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:21,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:21,116 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,195 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,203 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-11-19 23:53:21,228 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-11-19 23:53:21,240 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:21,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,262 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-11-19 23:53:21,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:21,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62588998, jitterRate=-0.06735125184059143}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-19 23:53:21,267 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-19 23:53:21,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-11-19 23:53:21,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:21,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:21,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:21,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-11-19 23:53:21,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 31 msec 2023-11-19 23:53:21,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-11-19 23:53:21,346 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-11-19 23:53:21,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-11-19 23:53:21,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-11-19 23:53:21,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-11-19 23:53:21,379 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-11-19 23:53:21,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-11-19 23:53:21,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-11-19 23:53:21,392 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:21,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-11-19 23:53:21,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-11-19 23:53:21,406 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-11-19 23:53:21,410 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-19 23:53:21,410 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-19 23:53:21,410 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-19 23:53:21,410 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-19 23:53:21,411 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:21,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,45383,1700437998977, sessionid=0x10008109e390000, setting cluster-up flag (Was=false) 2023-11-19 23:53:21,429 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:21,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-11-19 23:53:21,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:21,445 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:21,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-11-19 23:53:21,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:21,456 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.hbase-snapshot/.tmp 2023-11-19 23:53:21,472 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(951): ClusterId : 2701879b-bdc6-444e-ba28-c316ec4514dc 2023-11-19 23:53:21,472 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(951): ClusterId : 2701879b-bdc6-444e-ba28-c316ec4514dc 2023-11-19 23:53:21,472 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(951): ClusterId : 2701879b-bdc6-444e-ba28-c316ec4514dc 2023-11-19 23:53:21,477 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-19 23:53:21,477 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-19 23:53:21,477 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-19 23:53:21,484 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-19 23:53:21,484 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-19 23:53:21,484 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-19 23:53:21,484 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-19 23:53:21,484 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-19 23:53:21,484 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-19 23:53:21,488 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-19 23:53:21,488 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-19 23:53:21,488 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-19 23:53:21,491 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ReadOnlyZKClient(139): Connect 0x2b645a18 to 127.0.0.1:52135 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-19 23:53:21,492 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ReadOnlyZKClient(139): Connect 0x18f7687e to 127.0.0.1:52135 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-19 23:53:21,492 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ReadOnlyZKClient(139): Connect 0x029f2d0e to 127.0.0.1:52135 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-19 23:53:21,507 DEBUG [RS:1;jenkins-hbase4:34057] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@25a3b104, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-19 23:53:21,507 DEBUG [RS:1;jenkins-hbase4:34057] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@219c3168, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-19 23:53:21,509 DEBUG [RS:2;jenkins-hbase4:39613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16309510, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-19 23:53:21,509 DEBUG [RS:2;jenkins-hbase4:39613] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@120f030e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-19 23:53:21,510 DEBUG [RS:0;jenkins-hbase4:36283] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c58d32d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-19 23:53:21,511 DEBUG [RS:0;jenkins-hbase4:36283] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@867467e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-19 23:53:21,536 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36283 2023-11-19 23:53:21,536 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34057 2023-11-19 23:53:21,536 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:39613 2023-11-19 23:53:21,541 INFO [RS:0;jenkins-hbase4:36283] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-19 23:53:21,541 INFO [RS:2;jenkins-hbase4:39613] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-19 23:53:21,541 INFO [RS:2;jenkins-hbase4:39613] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-19 23:53:21,541 INFO [RS:1;jenkins-hbase4:34057] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-19 23:53:21,542 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1022): About to register with Master. 2023-11-19 23:53:21,541 INFO [RS:0;jenkins-hbase4:36283] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-19 23:53:21,542 INFO [RS:1;jenkins-hbase4:34057] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-19 23:53:21,542 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1022): About to register with Master. 2023-11-19 23:53:21,542 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1022): About to register with Master. 2023-11-19 23:53:21,545 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:36283, startcode=1700438000319 2023-11-19 23:53:21,545 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:39613, startcode=1700438000411 2023-11-19 23:53:21,545 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:34057, startcode=1700438000374 2023-11-19 23:53:21,568 DEBUG [RS:1;jenkins-hbase4:34057] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-19 23:53:21,568 DEBUG [RS:0;jenkins-hbase4:36283] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-19 23:53:21,568 DEBUG [RS:2;jenkins-hbase4:39613] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-19 23:53:21,619 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-19 23:53:21,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438031630 2023-11-19 23:53:21,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-11-19 23:53:21,632 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55475, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-11-19 23:53:21,632 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59001, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-11-19 23:53:21,633 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45129, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-11-19 23:53:21,639 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:21,639 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-11-19 23:53:21,645 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-11-19 23:53:21,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-11-19 23:53:21,649 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-19 23:53:21,657 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-11-19 23:53:21,657 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-11-19 23:53:21,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-11-19 23:53:21,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-11-19 23:53:21,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-11-19 23:53:21,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-11-19 23:53:21,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-11-19 23:53:21,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-11-19 23:53:21,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-11-19 23:53:21,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-11-19 23:53:21,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-11-19 23:53:21,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1700438001669,5,FailOnTimeoutGroup] 2023-11-19 23:53:21,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1700438001670,5,FailOnTimeoutGroup] 2023-11-19 23:53:21,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-11-19 23:53:21,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,680 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(2830): Master is not running yet 2023-11-19 23:53:21,680 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(2830): Master is not running yet 2023-11-19 23:53:21,680 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(2830): Master is not running yet 2023-11-19 23:53:21,680 WARN [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-11-19 23:53:21,680 WARN [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-11-19 23:53:21,680 WARN [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-11-19 23:53:21,704 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:21,705 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:21,705 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6 2023-11-19 23:53:21,727 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:21,729 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-19 23:53:21,732 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/info 2023-11-19 23:53:21,733 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-19 23:53:21,734 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:21,734 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-19 23:53:21,737 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/rep_barrier 2023-11-19 23:53:21,738 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-19 23:53:21,739 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:21,739 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-19 23:53:21,741 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/table 2023-11-19 23:53:21,742 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-19 23:53:21,743 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:21,745 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740 2023-11-19 23:53:21,746 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740 2023-11-19 23:53:21,750 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-19 23:53:21,752 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-19 23:53:21,759 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:21,760 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60241463, jitterRate=-0.10233224928379059}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-19 23:53:21,760 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-19 23:53:21,760 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-19 23:53:21,760 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-19 23:53:21,760 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-19 23:53:21,761 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-19 23:53:21,761 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-19 23:53:21,763 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-19 23:53:21,763 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-19 23:53:21,768 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:21,768 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-11-19 23:53:21,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-11-19 23:53:21,782 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:34057, startcode=1700438000374 2023-11-19 23:53:21,782 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:36283, startcode=1700438000319 2023-11-19 23:53:21,782 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,45383,1700437998977 with isa=jenkins-hbase4.apache.org/172.31.14.131:39613, startcode=1700438000411 2023-11-19 23:53:21,788 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,793 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-11-19 23:53:21,794 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,795 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6 2023-11-19 23:53:21,795 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45125 2023-11-19 23:53:21,795 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-11-19 23:53:21,795 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,797 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6 2023-11-19 23:53:21,797 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45125 2023-11-19 23:53:21,798 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-11-19 23:53:21,799 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6 2023-11-19 23:53:21,799 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:45125 2023-11-19 23:53:21,799 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-11-19 23:53:21,800 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-19 23:53:21,810 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:53:21,811 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ZKUtil(162): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,811 WARN [RS:2;jenkins-hbase4:39613] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-19 23:53:21,811 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ZKUtil(162): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,811 INFO [RS:2;jenkins-hbase4:39613] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-19 23:53:21,812 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,811 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ZKUtil(162): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,812 WARN [RS:1;jenkins-hbase4:34057] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-19 23:53:21,811 WARN [RS:0;jenkins-hbase4:36283] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-19 23:53:21,813 INFO [RS:0;jenkins-hbase4:36283] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-19 23:53:21,813 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,812 INFO [RS:1;jenkins-hbase4:34057] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-19 23:53:21,813 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34057,1700438000374] 2023-11-19 23:53:21,814 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1948): logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,814 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36283,1700438000319] 2023-11-19 23:53:21,814 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39613,1700438000411] 2023-11-19 23:53:21,826 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ZKUtil(162): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,826 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ZKUtil(162): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,826 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ZKUtil(162): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,827 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ZKUtil(162): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,827 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ZKUtil(162): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,827 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ZKUtil(162): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,828 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ZKUtil(162): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,828 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ZKUtil(162): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,829 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ZKUtil(162): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,839 DEBUG [RS:1;jenkins-hbase4:34057] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-19 23:53:21,839 DEBUG [RS:0;jenkins-hbase4:36283] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-19 23:53:21,839 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-19 23:53:21,848 INFO [RS:2;jenkins-hbase4:39613] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-19 23:53:21,848 INFO [RS:1;jenkins-hbase4:34057] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-19 23:53:21,848 INFO [RS:0;jenkins-hbase4:36283] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-19 23:53:21,867 INFO [RS:2;jenkins-hbase4:39613] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-19 23:53:21,867 INFO [RS:0;jenkins-hbase4:36283] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-19 23:53:21,867 INFO [RS:1;jenkins-hbase4:34057] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-19 23:53:21,872 INFO [RS:1;jenkins-hbase4:34057] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-19 23:53:21,872 INFO [RS:0;jenkins-hbase4:36283] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-19 23:53:21,872 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,872 INFO [RS:2;jenkins-hbase4:39613] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-19 23:53:21,873 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,873 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,876 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-11-19 23:53:21,876 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-11-19 23:53:21,876 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-11-19 23:53:21,884 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,884 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,884 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,884 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,884 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-19 23:53:21,886 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,885 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:1;jenkins-hbase4:34057] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,886 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,887 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,887 DEBUG [RS:0;jenkins-hbase4:36283] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,887 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-19 23:53:21,887 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,888 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,888 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,888 DEBUG [RS:2;jenkins-hbase4:39613] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-19 23:53:21,889 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,889 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,889 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,889 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,889 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,889 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,897 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,898 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,898 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,911 INFO [RS:1;jenkins-hbase4:34057] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-19 23:53:21,911 INFO [RS:2;jenkins-hbase4:39613] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-19 23:53:21,911 INFO [RS:0;jenkins-hbase4:36283] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-19 23:53:21,914 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36283,1700438000319-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,914 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39613,1700438000411-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,914 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34057,1700438000374-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:21,948 INFO [RS:2;jenkins-hbase4:39613] regionserver.Replication(203): jenkins-hbase4.apache.org,39613,1700438000411 started 2023-11-19 23:53:21,948 INFO [RS:0;jenkins-hbase4:36283] regionserver.Replication(203): jenkins-hbase4.apache.org,36283,1700438000319 started 2023-11-19 23:53:21,948 INFO [RS:1;jenkins-hbase4:34057] regionserver.Replication(203): jenkins-hbase4.apache.org,34057,1700438000374 started 2023-11-19 23:53:21,948 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36283,1700438000319, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36283, sessionid=0x10008109e390001 2023-11-19 23:53:21,948 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39613,1700438000411, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39613, sessionid=0x10008109e390003 2023-11-19 23:53:21,948 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34057,1700438000374, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34057, sessionid=0x10008109e390002 2023-11-19 23:53:21,949 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-19 23:53:21,949 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-19 23:53:21,949 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-19 23:53:21,949 DEBUG [RS:2;jenkins-hbase4:39613] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,949 DEBUG [RS:1;jenkins-hbase4:34057] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,949 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39613,1700438000411' 2023-11-19 23:53:21,949 DEBUG [RS:0;jenkins-hbase4:36283] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,950 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:21,950 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34057,1700438000374' 2023-11-19 23:53:21,950 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36283,1700438000319' 2023-11-19 23:53:21,950 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:21,950 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:21,951 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:21,951 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:21,951 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:21,951 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-19 23:53:21,951 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-19 23:53:21,952 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-19 23:53:21,952 DEBUG [RS:2;jenkins-hbase4:39613] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:21,952 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-19 23:53:21,952 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-19 23:53:21,952 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-19 23:53:21,952 DEBUG [RS:1;jenkins-hbase4:34057] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:21,953 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34057,1700438000374' 2023-11-19 23:53:21,953 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-19 23:53:21,952 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39613,1700438000411' 2023-11-19 23:53:21,953 DEBUG [jenkins-hbase4:45383] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-11-19 23:53:21,952 DEBUG [RS:0;jenkins-hbase4:36283] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:21,953 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-19 23:53:21,953 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36283,1700438000319' 2023-11-19 23:53:21,953 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-19 23:53:21,953 DEBUG [RS:1;jenkins-hbase4:34057] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-19 23:53:21,954 DEBUG [RS:2;jenkins-hbase4:39613] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-19 23:53:21,954 DEBUG [RS:0;jenkins-hbase4:36283] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-19 23:53:21,954 DEBUG [RS:1;jenkins-hbase4:34057] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-19 23:53:21,954 INFO [RS:1;jenkins-hbase4:34057] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-19 23:53:21,954 INFO [RS:1;jenkins-hbase4:34057] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-19 23:53:21,954 DEBUG [RS:2;jenkins-hbase4:39613] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-19 23:53:21,954 INFO [RS:2;jenkins-hbase4:39613] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-19 23:53:21,955 INFO [RS:2;jenkins-hbase4:39613] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-19 23:53:21,955 DEBUG [RS:0;jenkins-hbase4:36283] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-19 23:53:21,955 INFO [RS:0;jenkins-hbase4:36283] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-19 23:53:21,955 INFO [RS:0;jenkins-hbase4:36283] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-19 23:53:21,957 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:21,962 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:21,962 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:21,962 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:21,962 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:21,965 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39613,1700438000411, state=OPENING 2023-11-19 23:53:21,973 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-11-19 23:53:21,975 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:21,975 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-19 23:53:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:22,065 INFO [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39613%2C1700438000411, suffix=, logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,39613,1700438000411, archiveDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs, maxLogs=32 2023-11-19 23:53:22,065 INFO [RS:1;jenkins-hbase4:34057] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34057%2C1700438000374, suffix=, logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,34057,1700438000374, archiveDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs, maxLogs=32 2023-11-19 23:53:22,065 INFO [RS:0;jenkins-hbase4:36283] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36283%2C1700438000319, suffix=, logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,36283,1700438000319, archiveDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs, maxLogs=32 2023-11-19 23:53:22,089 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK] 2023-11-19 23:53:22,089 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK] 2023-11-19 23:53:22,090 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK] 2023-11-19 23:53:22,104 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK] 2023-11-19 23:53:22,104 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK] 2023-11-19 23:53:22,104 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK] 2023-11-19 23:53:22,107 INFO [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,39613,1700438000411/jenkins-hbase4.apache.org%2C39613%2C1700438000411.1700438002070 2023-11-19 23:53:22,110 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK] 2023-11-19 23:53:22,112 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK] 2023-11-19 23:53:22,112 DEBUG [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK], DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK]] 2023-11-19 23:53:22,118 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK] 2023-11-19 23:53:22,134 INFO [RS:0;jenkins-hbase4:36283] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,36283,1700438000319/jenkins-hbase4.apache.org%2C36283%2C1700438000319.1700438002071 2023-11-19 23:53:22,134 INFO [RS:1;jenkins-hbase4:34057] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,34057,1700438000374/jenkins-hbase4.apache.org%2C34057%2C1700438000374.1700438002071 2023-11-19 23:53:22,135 DEBUG [RS:0;jenkins-hbase4:36283] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK], DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK]] 2023-11-19 23:53:22,136 DEBUG [RS:1;jenkins-hbase4:34057] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK], DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK]] 2023-11-19 23:53:22,163 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:22,165 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-19 23:53:22,168 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57674, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-19 23:53:22,180 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-11-19 23:53:22,181 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-19 23:53:22,184 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39613%2C1700438000411.meta, suffix=.meta, logDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,39613,1700438000411, archiveDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs, maxLogs=32 2023-11-19 23:53:22,203 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK] 2023-11-19 23:53:22,205 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK] 2023-11-19 23:53:22,205 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK] 2023-11-19 23:53:22,210 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/WALs/jenkins-hbase4.apache.org,39613,1700438000411/jenkins-hbase4.apache.org%2C39613%2C1700438000411.meta.1700438002186.meta 2023-11-19 23:53:22,211 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36193,DS-6ff88d8f-151e-4c55-a58e-9aef0e06132b,DISK], DatanodeInfoWithStorage[127.0.0.1:44679,DS-066cee74-6470-4910-b337-571812d566b1,DISK], DatanodeInfoWithStorage[127.0.0.1:42551,DS-2897e77b-c5c0-4723-a296-ada3a640a3dd,DISK]] 2023-11-19 23:53:22,211 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:22,215 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:22,216 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-11-19 23:53:22,232 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-11-19 23:53:22,234 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-11-19 23:53:22,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-11-19 23:53:22,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:22,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-11-19 23:53:22,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-11-19 23:53:22,241 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-19 23:53:22,243 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/info 2023-11-19 23:53:22,243 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/info 2023-11-19 23:53:22,245 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-19 23:53:22,247 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:22,247 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-19 23:53:22,249 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/rep_barrier 2023-11-19 23:53:22,249 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/rep_barrier 2023-11-19 23:53:22,250 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-19 23:53:22,251 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:22,251 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-19 23:53:22,252 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/table 2023-11-19 23:53:22,253 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/table 2023-11-19 23:53:22,253 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-19 23:53:22,254 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:22,255 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740 2023-11-19 23:53:22,258 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740 2023-11-19 23:53:22,261 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-19 23:53:22,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-19 23:53:22,265 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67944568, jitterRate=0.012452960014343262}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-19 23:53:22,265 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-19 23:53:22,276 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1700438002155 2023-11-19 23:53:22,292 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-11-19 23:53:22,293 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-11-19 23:53:22,293 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39613,1700438000411, state=OPEN 2023-11-19 23:53:22,296 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-11-19 23:53:22,296 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-19 23:53:22,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-11-19 23:53:22,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39613,1700438000411 in 318 msec 2023-11-19 23:53:22,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-11-19 23:53:22,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 525 msec 2023-11-19 23:53:22,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 787 msec 2023-11-19 23:53:22,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1700438002311, completionTime=-1 2023-11-19 23:53:22,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-11-19 23:53:22,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:22,379 DEBUG [hconnection-0x4b9cda40-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-19 23:53:22,382 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-19 23:53:22,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:22,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438062399 2023-11-19 23:53:22,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438122399 2023-11-19 23:53:22,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 87 msec 2023-11-19 23:53:22,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45383,1700437998977-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:22,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45383,1700437998977-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:22,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45383,1700437998977-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:22,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:45383, period=300000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:22,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-11-19 23:53:22,436 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-11-19 23:53:22,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-11-19 23:53:22,448 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-19 23:53:22,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:22,464 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:22,467 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:22,489 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/hbase/namespace/329994419c4b206406658a465b604c17 2023-11-19 23:53:22,492 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/hbase/namespace/329994419c4b206406658a465b604c17 empty. 2023-11-19 23:53:22,492 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/hbase/namespace/329994419c4b206406658a465b604c17 2023-11-19 23:53:22,492 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-11-19 23:53:22,526 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:22,528 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 329994419c4b206406658a465b604c17, NAME => 'hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 329994419c4b206406658a465b604c17, disabling compactions & flushes 2023-11-19 23:53:22,543 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. after waiting 0 ms 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,543 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,543 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 329994419c4b206406658a465b604c17: 2023-11-19 23:53:22,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:22,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1700438002550"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438002550"}]},"ts":"1700438002550"} 2023-11-19 23:53:22,589 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:22,591 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:22,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438002591"}]},"ts":"1700438002591"} 2023-11-19 23:53:22,600 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-11-19 23:53:22,606 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:22,607 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:22,607 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:22,607 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:22,607 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:22,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=329994419c4b206406658a465b604c17, ASSIGN}] 2023-11-19 23:53:22,613 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=329994419c4b206406658a465b604c17, ASSIGN 2023-11-19 23:53:22,614 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=329994419c4b206406658a465b604c17, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:22,766 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:22,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=329994419c4b206406658a465b604c17, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:22,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1700438002767"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438002767"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438002767"}]},"ts":"1700438002767"} 2023-11-19 23:53:22,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 329994419c4b206406658a465b604c17, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:22,931 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:22,941 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,943 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 329994419c4b206406658a465b604c17, NAME => 'hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:22,945 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:22,945 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 329994419c4b206406658a465b604c17 2023-11-19 23:53:22,946 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:22,946 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 329994419c4b206406658a465b604c17 2023-11-19 23:53:22,946 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 329994419c4b206406658a465b604c17 2023-11-19 23:53:22,948 INFO [StoreOpener-329994419c4b206406658a465b604c17-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 329994419c4b206406658a465b604c17 2023-11-19 23:53:22,951 DEBUG [StoreOpener-329994419c4b206406658a465b604c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/info 2023-11-19 23:53:22,951 DEBUG [StoreOpener-329994419c4b206406658a465b604c17-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/info 2023-11-19 23:53:22,951 INFO [StoreOpener-329994419c4b206406658a465b604c17-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 329994419c4b206406658a465b604c17 columnFamilyName info 2023-11-19 23:53:22,952 INFO [StoreOpener-329994419c4b206406658a465b604c17-1] regionserver.HStore(310): Store=329994419c4b206406658a465b604c17/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:22,954 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17 2023-11-19 23:53:22,955 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17 2023-11-19 23:53:22,960 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 329994419c4b206406658a465b604c17 2023-11-19 23:53:22,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:22,965 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 329994419c4b206406658a465b604c17; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72579067, jitterRate=0.08151237666606903}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-19 23:53:22,965 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 329994419c4b206406658a465b604c17: 2023-11-19 23:53:22,967 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17., pid=6, masterSystemTime=1700438002931 2023-11-19 23:53:22,971 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,971 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:53:22,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=329994419c4b206406658a465b604c17, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:22,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1700438002972"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438002972"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438002972"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438002972"}]},"ts":"1700438002972"} 2023-11-19 23:53:22,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-11-19 23:53:22,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 329994419c4b206406658a465b604c17, server=jenkins-hbase4.apache.org,39613,1700438000411 in 199 msec 2023-11-19 23:53:22,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-11-19 23:53:22,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=329994419c4b206406658a465b604c17, ASSIGN in 372 msec 2023-11-19 23:53:22,986 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:22,986 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438002986"}]},"ts":"1700438002986"} 2023-11-19 23:53:22,989 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-11-19 23:53:22,992 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:22,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 543 msec 2023-11-19 23:53:23,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-11-19 23:53:23,066 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-11-19 23:53:23,066 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:23,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:23,117 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-19 23:53:23,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-11-19 23:53:23,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:23,146 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-19 23:53:23,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-11-19 23:53:23,161 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-11-19 23:53:23,165 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-11-19 23:53:23,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.654sec 2023-11-19 23:53:23,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-11-19 23:53:23,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-11-19 23:53:23,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-11-19 23:53:23,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45383,1700437998977-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-11-19 23:53:23,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45383,1700437998977-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-11-19 23:53:23,178 DEBUG [Listener at localhost/42385] zookeeper.ReadOnlyZKClient(139): Connect 0x7b833cd0 to 127.0.0.1:52135 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-19 23:53:23,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-11-19 23:53:23,183 DEBUG [Listener at localhost/42385] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c47ee48, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-19 23:53:23,197 DEBUG [hconnection-0x329d2ba3-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-19 23:53:23,211 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47980, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-19 23:53:23,222 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:53:23,255 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=723, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=172, AvailableMemoryMB=15494 2023-11-19 23:53:23,260 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:23,269 DEBUG [Listener at localhost/42385] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-19 23:53:23,272 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38756, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-19 23:53:23,288 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:23,288 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:53:23,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:23,294 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,296 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:23,298 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:23,301 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,301 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b empty. 2023-11-19 23:53:23,303 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,303 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-11-19 23:53:23,323 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:23,326 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => dae2bb8e46806b529da56f5c659c8d9b, NAME => 'testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing dae2bb8e46806b529da56f5c659c8d9b, disabling compactions & flushes 2023-11-19 23:53:23,345 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. after waiting 0 ms 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,345 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,345 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:23,349 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:23,350 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438003350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438003350"}]},"ts":"1700438003350"} 2023-11-19 23:53:23,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:23,354 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:23,354 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438003354"}]},"ts":"1700438003354"} 2023-11-19 23:53:23,356 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-11-19 23:53:23,361 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:23,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:23,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:23,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:23,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:23,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, ASSIGN}] 2023-11-19 23:53:23,365 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, ASSIGN 2023-11-19 23:53:23,366 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:23,516 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:23,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=dae2bb8e46806b529da56f5c659c8d9b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:23,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438003517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438003517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438003517"}]},"ts":"1700438003517"} 2023-11-19 23:53:23,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure dae2bb8e46806b529da56f5c659c8d9b, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:23,673 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:23,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dae2bb8e46806b529da56f5c659c8d9b, NAME => 'testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:23,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:23,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:23,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,683 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,685 DEBUG [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1 2023-11-19 23:53:23,686 DEBUG [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1 2023-11-19 23:53:23,686 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dae2bb8e46806b529da56f5c659c8d9b columnFamilyName cf1 2023-11-19 23:53:23,687 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] regionserver.HStore(310): Store=dae2bb8e46806b529da56f5c659c8d9b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:23,687 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,689 DEBUG [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2 2023-11-19 23:53:23,689 DEBUG [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2 2023-11-19 23:53:23,690 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dae2bb8e46806b529da56f5c659c8d9b columnFamilyName cf2 2023-11-19 23:53:23,691 INFO [StoreOpener-dae2bb8e46806b529da56f5c659c8d9b-1] regionserver.HStore(310): Store=dae2bb8e46806b529da56f5c659c8d9b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:23,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:23,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:23,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:23,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dae2bb8e46806b529da56f5c659c8d9b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62420661, jitterRate=-0.06985966861248016}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:23,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:23,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b., pid=11, masterSystemTime=1700438003673 2023-11-19 23:53:23,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=dae2bb8e46806b529da56f5c659c8d9b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:23,709 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438003709"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438003709"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438003709"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438003709"}]},"ts":"1700438003709"} 2023-11-19 23:53:23,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-11-19 23:53:23,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure dae2bb8e46806b529da56f5c659c8d9b, server=jenkins-hbase4.apache.org,39613,1700438000411 in 191 msec 2023-11-19 23:53:23,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-11-19 23:53:23,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, ASSIGN in 356 msec 2023-11-19 23:53:23,722 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:23,722 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438003722"}]},"ts":"1700438003722"} 2023-11-19 23:53:23,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-11-19 23:53:23,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:23,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 435 msec 2023-11-19 23:53:23,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:23,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,825 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:23,825 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:23,825 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:23,825 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:23,826 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,826 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,828 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:23,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:23,828 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:23,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:23,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:23,828 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:23,829 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:23,833 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:23,834 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:23,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:23,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:23,835 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:23,835 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:23,836 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:53:23,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,841 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:23,842 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:23,842 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:23,909 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/b685e341e30a4bd281afa6532d30102e 2023-11-19 23:53:23,966 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/676484a783d144a098ddf008fba19755 2023-11-19 23:53:23,974 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/b685e341e30a4bd281afa6532d30102e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e 2023-11-19 23:53:23,983 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:23,985 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/676484a783d144a098ddf008fba19755 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755 2023-11-19 23:53:23,994 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:23,997 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 161ms, sequenceid=6, compaction requested=false 2023-11-19 23:53:23,998 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,000 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:24,000 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,000 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,004 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,004 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,004 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,004 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:24,008 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:24,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31b5052b[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:24,008 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,011 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,012 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,012 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:24,012 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:24,019 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,020 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,021 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,021 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,021 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:24,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:24,028 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:24,028 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:24,028 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:24,028 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,028 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:24,031 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,031 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,031 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,031 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,032 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,034 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,035 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,036 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,041 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,042 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,043 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,069 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,070 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,070 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,070 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:24,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,069 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,073 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:24,073 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:24,173 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:24,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:24,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:24,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:24,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,206 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,206 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,206 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:24,207 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:24,207 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,207 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,210 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,210 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,211 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,211 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,213 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,213 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,213 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,215 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,215 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,215 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,215 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,215 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,216 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,216 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,216 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,216 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,217 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:24,217 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:24,217 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,217 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:24,217 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:24,218 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,219 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,219 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,220 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,220 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,220 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,237 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/e30832324f8a4f02a1df76dba688043c 2023-11-19 23:53:24,263 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/9bbd751b99334713960e8822136c5b1a 2023-11-19 23:53:24,271 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/e30832324f8a4f02a1df76dba688043c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c 2023-11-19 23:53:24,280 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:24,281 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/9bbd751b99334713960e8822136c5b1a as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a 2023-11-19 23:53:24,289 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:24,290 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 73ms, sequenceid=19, compaction requested=false 2023-11-19 23:53:24,290 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:24,290 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:24,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:24,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,295 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:24,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f002d3e[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:24,298 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:24,298 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,300 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:24,300 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,301 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:24,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:24,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,306 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,306 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,306 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,307 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,308 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,308 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,308 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:24,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:24,314 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:24,314 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:24,314 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:24,314 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,314 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,317 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,318 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,319 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,322 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,322 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,323 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,330 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,331 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,331 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,331 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:24,331 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,332 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:24,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,432 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:24,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:24,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:24,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:24,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,468 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,468 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,469 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:24,469 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:24,469 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,469 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,471 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,471 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,471 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,471 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,472 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,473 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,473 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,474 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,474 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,475 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,475 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,475 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,476 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:24,477 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:24,477 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,478 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:24,478 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:24,479 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,480 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,480 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,482 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,508 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/03e59751fba34aedbbec7b142ac93238 2023-11-19 23:53:24,547 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/b7f89e290baa4633b27e8a86bb72fe36 2023-11-19 23:53:24,557 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/03e59751fba34aedbbec7b142ac93238 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238 2023-11-19 23:53:24,566 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:24,567 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/b7f89e290baa4633b27e8a86bb72fe36 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36 2023-11-19 23:53:24,576 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:24,577 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 99ms, sequenceid=32, compaction requested=true 2023-11-19 23:53:24,577 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:24,577 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,577 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,578 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,581 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,581 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,582 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,582 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:24,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@164aecb6[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:24,585 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:24,585 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,588 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:24,588 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,588 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:24,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:24,589 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:24,592 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,592 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,593 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,593 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,593 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,593 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,593 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,594 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,594 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,594 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:24,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:24,609 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:24,609 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:24,609 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:24,609 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,609 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:24,611 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,611 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,611 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,612 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,612 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,612 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,613 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,614 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,618 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,619 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,620 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,628 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,629 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,629 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-11-19 23:53:24,630 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,730 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:24,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:24,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:24,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:24,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,757 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,757 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,758 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:24,758 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:24,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,760 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,760 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,760 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,760 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,761 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,761 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,764 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,764 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,764 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,765 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,765 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,765 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:24,766 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,765 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,766 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,766 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,766 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:24,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,766 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:24,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:24,766 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,767 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:24,768 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:24,770 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,771 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,771 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,771 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,771 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,771 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,786 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/515a0de064de4ff1b67559213cca8e40 2023-11-19 23:53:24,810 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/6261b2d1f4a2425abb0082980775c56e 2023-11-19 23:53:24,818 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/515a0de064de4ff1b67559213cca8e40 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40 2023-11-19 23:53:24,825 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:24,826 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/6261b2d1f4a2425abb0082980775c56e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e 2023-11-19 23:53:24,834 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:24,835 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 68ms, sequenceid=45, compaction requested=true 2023-11-19 23:53:24,835 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:24,835 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:24,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:24,836 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:24,836 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,836 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:24,836 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:24,839 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,839 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,840 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,840 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:24,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15271eb[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:24,843 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:24,843 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,844 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,844 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:24,845 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,848 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,848 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:24,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:24,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:24,854 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:24,854 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:24,854 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:24,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:24,854 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,855 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:24,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,858 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:24,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,859 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,860 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:24,862 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,863 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,863 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:24,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,869 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:24,869 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:24,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,868 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,869 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:24,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:24,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,870 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:24,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:24,970 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:24,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:24,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:24,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:24,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,995 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:24,996 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:24,996 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:24,996 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:24,997 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,997 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,998 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:24,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,999 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:24,999 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:24,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,001 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,001 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,001 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,002 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,003 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,003 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,003 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,003 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,004 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,005 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:25,005 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:25,005 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,005 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:25,005 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:25,006 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,008 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,008 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,009 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,009 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,009 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,023 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/7adc0a4e673a4221b6eae09d706cd16f 2023-11-19 23:53:25,048 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/cd4eef954cd74c90b8dc1df4119fab5a 2023-11-19 23:53:25,056 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/7adc0a4e673a4221b6eae09d706cd16f as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f 2023-11-19 23:53:25,063 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:25,065 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/cd4eef954cd74c90b8dc1df4119fab5a as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a 2023-11-19 23:53:25,073 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:25,074 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 69ms, sequenceid=58, compaction requested=true 2023-11-19 23:53:25,074 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:25,074 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,074 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:25,075 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:25,075 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,075 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,075 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,078 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,078 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,078 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,078 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:25,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18d7a0b7[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:25,081 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:25,082 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,085 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,085 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,085 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,085 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:25,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,086 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,090 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,090 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,091 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,092 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:25,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:25,098 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:25,098 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:25,098 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:25,098 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,098 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:25,100 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,100 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,104 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,106 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,106 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,107 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,111 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,111 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,111 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,111 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,111 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,112 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:25,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:25,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,214 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:25,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:25,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:25,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:25,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,239 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:25,239 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-11-19 23:53:25,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-11-19 23:53:25,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,243 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,243 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,243 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,243 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,243 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,245 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,245 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,245 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,246 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:25,246 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:25,246 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. started... 2023-11-19 23:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,246 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:25,247 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,247 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,250 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,250 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,251 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,251 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,251 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,251 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,264 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/6214522d85fe411aac06d486f0810868 2023-11-19 23:53:25,286 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/50879b95790f4a3d97148c1665b95021 2023-11-19 23:53:25,293 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/6214522d85fe411aac06d486f0810868 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868 2023-11-19 23:53:25,300 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:25,301 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/50879b95790f4a3d97148c1665b95021 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021 2023-11-19 23:53:25,308 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:25,309 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 62ms, sequenceid=71, compaction requested=true 2023-11-19 23:53:25,309 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:25,309 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,309 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:25,311 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:25,311 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,311 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-11-19 23:53:25,311 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-11-19 23:53:25,313 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,313 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,314 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,314 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:25,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-11-19 23:53:25,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5670074e[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:25,316 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:25,316 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,318 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:25,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:25,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-11-19 23:53:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-11-19 23:53:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:25,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,321 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,322 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-11-19 23:53:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-11-19 23:53:25,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:25,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:25,328 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-11-19 23:53:25,328 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:25,328 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:25,328 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,328 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,333 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,333 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:25,334 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:25,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,337 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,337 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,342 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,342 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:25,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-11-19 23:53:25,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:25,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,343 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:25,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:25,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-11-19 23:53:25,344 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:25,345 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:25,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:25,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:25,445 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:25,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:25,466 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,473 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:25,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:25,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:25,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:25,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, UNASSIGN}] 2023-11-19 23:53:25,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, UNASSIGN 2023-11-19 23:53:25,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=dae2bb8e46806b529da56f5c659c8d9b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438005483"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438005483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438005483"}]},"ts":"1700438005483"} 2023-11-19 23:53:25,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure dae2bb8e46806b529da56f5c659c8d9b, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:25,641 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:25,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dae2bb8e46806b529da56f5c659c8d9b, disabling compactions & flushes 2023-11-19 23:53:25,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. after waiting 0 ms 2023-11-19 23:53:25,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing dae2bb8e46806b529da56f5c659c8d9b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:53:25,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/697ffcb6452c4c6eaf6b97a429c8c041 2023-11-19 23:53:25,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/0424de1a738546a98f618bf1559b23d2 2023-11-19 23:53:25,690 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf1/697ffcb6452c4c6eaf6b97a429c8c041 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041 2023-11-19 23:53:25,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:25,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.tmp/cf2/0424de1a738546a98f618bf1559b23d2 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2 2023-11-19 23:53:25,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:25,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for dae2bb8e46806b529da56f5c659c8d9b in 62ms, sequenceid=82, compaction requested=true 2023-11-19 23:53:25,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:53:25,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:25,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. 2023-11-19 23:53:25,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dae2bb8e46806b529da56f5c659c8d9b: 2023-11-19 23:53:25,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=dae2bb8e46806b529da56f5c659c8d9b, regionState=CLOSED 2023-11-19 23:53:25,738 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438005737"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438005737"}]},"ts":"1700438005737"} 2023-11-19 23:53:25,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-11-19 23:53:25,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure dae2bb8e46806b529da56f5c659c8d9b, server=jenkins-hbase4.apache.org,39613,1700438000411 in 255 msec 2023-11-19 23:53:25,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=13, resume processing ppid=12 2023-11-19 23:53:25,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=dae2bb8e46806b529da56f5c659c8d9b, UNASSIGN in 266 msec 2023-11-19 23:53:25,770 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=dae2bb8e46806b529da56f5c659c8d9b, threads=14 2023-11-19 23:53:25,770 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,771 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,772 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,772 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,772 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,772 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,816 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2, top=true 2023-11-19 23:53:25,817 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f, top=true 2023-11-19 23:53:25,818 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021, top=true 2023-11-19 23:53:25,840 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a, top=true 2023-11-19 23:53:25,841 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868, top=true 2023-11-19 23:53:25,842 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041, top=true 2023-11-19 23:53:25,875 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2 for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,875 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,876 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,875 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,888 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868 for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,888 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,906 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,906 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,908 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021 for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,915 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,929 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/.splits/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041 for child: 31dbf9d0416d92b720428ba12f6265d3, parent: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,929 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,957 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,967 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,967 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,979 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,979 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,988 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:25,989 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40 for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:26,329 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e for region: dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:26,329 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region dae2bb8e46806b529da56f5c659c8d9b Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:26,397 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:26,399 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:26,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1700438006402"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1700438006402"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1700438006402"}]},"ts":"1700438006402"} 2023-11-19 23:53:26,403 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438006402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438006402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438006402"}]},"ts":"1700438006402"} 2023-11-19 23:53:26,403 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438006402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438006402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438006402"}]},"ts":"1700438006402"} 2023-11-19 23:53:26,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, ASSIGN}] 2023-11-19 23:53:26,465 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, ASSIGN 2023-11-19 23:53:26,465 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:26,466 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, ASSIGN 2023-11-19 23:53:26,467 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:26,616 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:53:26,617 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5b5dc1190d203ff79365b9903c40512a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:26,617 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438006617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438006617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438006617"}]},"ts":"1700438006617"} 2023-11-19 23:53:26,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 5b5dc1190d203ff79365b9903c40512a, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:26,624 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=31dbf9d0416d92b720428ba12f6265d3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:26,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438006623"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438006623"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438006623"}]},"ts":"1700438006623"} 2023-11-19 23:53:26,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 31dbf9d0416d92b720428ba12f6265d3, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:26,772 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:26,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 31dbf9d0416d92b720428ba12f6265d3, NAME => 'testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.', STARTKEY => '26', ENDKEY => ''} 2023-11-19 23:53:26,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:26,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:26,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,779 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,780 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1 2023-11-19 23:53:26,780 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1 2023-11-19 23:53:26,781 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31dbf9d0416d92b720428ba12f6265d3 columnFamilyName cf1 2023-11-19 23:53:26,815 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-top 2023-11-19 23:53:26,822 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-top 2023-11-19 23:53:26,828 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-top 2023-11-19 23:53:26,835 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-top 2023-11-19 23:53:26,841 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868 2023-11-19 23:53:26,846 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041 2023-11-19 23:53:26,852 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f 2023-11-19 23:53:26,852 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(310): Store=31dbf9d0416d92b720428ba12f6265d3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:26,852 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,854 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2 2023-11-19 23:53:26,854 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2 2023-11-19 23:53:26,854 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31dbf9d0416d92b720428ba12f6265d3 columnFamilyName cf2 2023-11-19 23:53:26,869 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-top 2023-11-19 23:53:26,874 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-top 2023-11-19 23:53:26,880 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-top 2023-11-19 23:53:26,886 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-top 2023-11-19 23:53:26,891 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2 2023-11-19 23:53:26,896 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021 2023-11-19 23:53:26,901 DEBUG [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a 2023-11-19 23:53:26,901 INFO [StoreOpener-31dbf9d0416d92b720428ba12f6265d3-1] regionserver.HStore(310): Store=31dbf9d0416d92b720428ba12f6265d3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:26,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:26,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:26,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 31dbf9d0416d92b720428ba12f6265d3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61019366, jitterRate=-0.09074059128761292}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:26,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 31dbf9d0416d92b720428ba12f6265d3: 2023-11-19 23:53:26,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3., pid=18, masterSystemTime=1700438006772 2023-11-19 23:53:26,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:26,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:26,914 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:26,914 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:26,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,916 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,916 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1912): 31dbf9d0416d92b720428ba12f6265d3/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:26,916 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:26,917 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1912): 31dbf9d0416d92b720428ba12f6265d3/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:26,917 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 31dbf9d0416d92b720428ba12f6265d3/cf2 in testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5b5dc1190d203ff79365b9903c40512a, NAME => 'testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.', STARTKEY => '', ENDKEY => '26'} 2023-11-19 23:53:26,917 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 31dbf9d0416d92b720428ba12f6265d3/cf1 in testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:26,917 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=31dbf9d0416d92b720428ba12f6265d3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:26,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:26,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:26,917 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438006917"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438006917"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438006917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438006917"}]},"ts":"1700438006917"} 2023-11-19 23:53:26,917 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/.tmp, totalSize=34.9 K 2023-11-19 23:53:26,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:26,918 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/.tmp, totalSize=34.9 K 2023-11-19 23:53:26,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:26,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:26,920 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438003800 2023-11-19 23:53:26,921 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438003800 2023-11-19 23:53:26,922 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438004181 2023-11-19 23:53:26,922 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:26,923 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438004181 2023-11-19 23:53:26,923 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438004435 2023-11-19 23:53:26,923 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438004435 2023-11-19 23:53:26,924 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438004732 2023-11-19 23:53:26,925 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1 2023-11-19 23:53:26,925 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438004973 2023-11-19 23:53:26,925 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1 2023-11-19 23:53:26,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=18, resume processing ppid=16 2023-11-19 23:53:26,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 31dbf9d0416d92b720428ba12f6265d3, server=jenkins-hbase4.apache.org,39613,1700438000411 in 297 msec 2023-11-19 23:53:26,926 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438005217 2023-11-19 23:53:26,926 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5b5dc1190d203ff79365b9903c40512a columnFamilyName cf1 2023-11-19 23:53:26,925 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438004732 2023-11-19 23:53:26,927 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438005447 2023-11-19 23:53:26,928 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438004973 2023-11-19 23:53:26,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, ASSIGN in 462 msec 2023-11-19 23:53:26,929 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438005217 2023-11-19 23:53:26,930 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438005447 2023-11-19 23:53:26,943 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-bottom 2023-11-19 23:53:26,963 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-bottom 2023-11-19 23:53:27,010 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] throttle.PressureAwareThroughputController(145): 31dbf9d0416d92b720428ba12f6265d3#cf1#compaction#14 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:27,014 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): 31dbf9d0416d92b720428ba12f6265d3#cf2#compaction#15 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:27,025 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-bottom 2023-11-19 23:53:27,031 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-bottom 2023-11-19 23:53:27,032 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(310): Store=5b5dc1190d203ff79365b9903c40512a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:27,033 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,034 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2 2023-11-19 23:53:27,034 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2 2023-11-19 23:53:27,035 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5b5dc1190d203ff79365b9903c40512a columnFamilyName cf2 2023-11-19 23:53:27,053 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-bottom 2023-11-19 23:53:27,064 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-bottom 2023-11-19 23:53:27,069 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/.tmp/cf2/22948d3cec6b4c72b997ff3595353063 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/22948d3cec6b4c72b997ff3595353063 2023-11-19 23:53:27,073 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-bottom 2023-11-19 23:53:27,082 DEBUG [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-bottom 2023-11-19 23:53:27,082 INFO [StoreOpener-5b5dc1190d203ff79365b9903c40512a-1] regionserver.HStore(310): Store=5b5dc1190d203ff79365b9903c40512a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:27,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,086 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/.tmp/cf1/7a4e03f73e0a4d4f94be3343cfa391b5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/7a4e03f73e0a4d4f94be3343cfa391b5 2023-11-19 23:53:27,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:27,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5b5dc1190d203ff79365b9903c40512a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73006968, jitterRate=0.08788859844207764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:27,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5b5dc1190d203ff79365b9903c40512a: 2023-11-19 23:53:27,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a., pid=17, masterSystemTime=1700438006772 2023-11-19 23:53:27,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:27,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-19 23:53:27,093 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 31dbf9d0416d92b720428ba12f6265d3/cf2 of 31dbf9d0416d92b720428ba12f6265d3 into 22948d3cec6b4c72b997ff3595353063(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:27,094 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 31dbf9d0416d92b720428ba12f6265d3: 2023-11-19 23:53:27,094 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3., storeName=31dbf9d0416d92b720428ba12f6265d3/cf2, priority=9, startTime=1700438006914; duration=0sec 2023-11-19 23:53:27,094 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 31dbf9d0416d92b720428ba12f6265d3/cf1 of 31dbf9d0416d92b720428ba12f6265d3 into 7a4e03f73e0a4d4f94be3343cfa391b5(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:27,094 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 31dbf9d0416d92b720428ba12f6265d3: 2023-11-19 23:53:27,094 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3., storeName=31dbf9d0416d92b720428ba12f6265d3/cf1, priority=9, startTime=1700438006913; duration=0sec 2023-11-19 23:53:27,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,094 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-19 23:53:27,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,094 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-19 23:53:27,095 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-19 23:53:27,095 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-19 23:53:27,095 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,095 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,095 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1912): 5b5dc1190d203ff79365b9903c40512a/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:27,095 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1912): 5b5dc1190d203ff79365b9903c40512a/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:27,096 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 5b5dc1190d203ff79365b9903c40512a/cf1 in testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,096 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5b5dc1190d203ff79365b9903c40512a/cf2 in testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,096 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5b5dc1190d203ff79365b9903c40512a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:27,096 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438007096"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438007096"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438007096"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438007096"}]},"ts":"1700438007096"} 2023-11-19 23:53:27,096 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/.tmp, totalSize=19.9 K 2023-11-19 23:53:27,096 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/.tmp, totalSize=19.9 K 2023-11-19 23:53:27,098 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438003800 2023-11-19 23:53:27,098 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438003800 2023-11-19 23:53:27,099 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438004181 2023-11-19 23:53:27,100 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438004181 2023-11-19 23:53:27,100 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438004435 2023-11-19 23:53:27,100 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438004435 2023-11-19 23:53:27,101 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438004732 2023-11-19 23:53:27,101 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438004732 2023-11-19 23:53:27,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-11-19 23:53:27,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 5b5dc1190d203ff79365b9903c40512a, server=jenkins-hbase4.apache.org,39613,1700438000411 in 480 msec 2023-11-19 23:53:27,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=15, resume processing ppid=12 2023-11-19 23:53:27,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, ASSIGN in 640 msec 2023-11-19 23:53:27,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 in 1.6390 sec 2023-11-19 23:53:27,114 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): 5b5dc1190d203ff79365b9903c40512a#cf1#compaction#17 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:27,115 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5b5dc1190d203ff79365b9903c40512a#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:27,135 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/.tmp/cf2/1fd316cac7e64a218592ae28a4e651f4 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/1fd316cac7e64a218592ae28a4e651f4 2023-11-19 23:53:27,137 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/.tmp/cf1/ed536f2d67ce463a966629b3e01e6e44 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/ed536f2d67ce463a966629b3e01e6e44 2023-11-19 23:53:27,143 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5b5dc1190d203ff79365b9903c40512a/cf2 of 5b5dc1190d203ff79365b9903c40512a into 1fd316cac7e64a218592ae28a4e651f4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:27,144 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5b5dc1190d203ff79365b9903c40512a: 2023-11-19 23:53:27,144 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a., storeName=5b5dc1190d203ff79365b9903c40512a/cf2, priority=12, startTime=1700438007091; duration=0sec 2023-11-19 23:53:27,144 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:27,145 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5b5dc1190d203ff79365b9903c40512a/cf1 of 5b5dc1190d203ff79365b9903c40512a into ed536f2d67ce463a966629b3e01e6e44(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:27,146 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5b5dc1190d203ff79365b9903c40512a: 2023-11-19 23:53:27,146 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a., storeName=5b5dc1190d203ff79365b9903c40512a/cf1, priority=12, startTime=1700438007091; duration=0sec 2023-11-19 23:53:27,146 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:27,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39613] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:47980 deadline: 1700438067267, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b. is not online on jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:27,384 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-11-19 23:53:27,394 INFO [Listener at localhost/42385] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-11-19 23:53:27,397 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-11-19 23:53:27,426 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:27,435 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,449 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438007449"}]},"ts":"1700438007449"} 2023-11-19 23:53:27,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-11-19 23:53:27,453 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-11-19 23:53:27,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-19 23:53:27,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, UNASSIGN}] 2023-11-19 23:53:27,457 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, UNASSIGN 2023-11-19 23:53:27,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=31dbf9d0416d92b720428ba12f6265d3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:27,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438007458"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438007458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438007458"}]},"ts":"1700438007458"} 2023-11-19 23:53:27,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 31dbf9d0416d92b720428ba12f6265d3, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:27,463 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, UNASSIGN 2023-11-19 23:53:27,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5b5dc1190d203ff79365b9903c40512a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:27,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438007464"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438007464"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438007464"}]},"ts":"1700438007464"} 2023-11-19 23:53:27,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 5b5dc1190d203ff79365b9903c40512a, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:27,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-19 23:53:27,612 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:27,612 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5b5dc1190d203ff79365b9903c40512a, disabling compactions & flushes 2023-11-19 23:53:27,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. after waiting 0 ms 2023-11-19 23:53:27,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,620 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-bottom] to archive 2023-11-19 23:53:27,622 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:27,632 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,635 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,637 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,640 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,667 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-bottom] to archive 2023-11-19 23:53:27,669 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:27,671 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,673 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,674 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,676 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:53:27,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:27,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a. 2023-11-19 23:53:27,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5b5dc1190d203ff79365b9903c40512a: 2023-11-19 23:53:27,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:27,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 31dbf9d0416d92b720428ba12f6265d3, disabling compactions & flushes 2023-11-19 23:53:27,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:27,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:27,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. after waiting 0 ms 2023-11-19 23:53:27,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:27,686 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5b5dc1190d203ff79365b9903c40512a, regionState=CLOSED 2023-11-19 23:53:27,688 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438007686"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438007686"}]},"ts":"1700438007686"} 2023-11-19 23:53:27,689 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041] to archive 2023-11-19 23:53:27,691 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:27,693 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/b685e341e30a4bd281afa6532d30102e.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=23, resume processing ppid=20 2023-11-19 23:53:27,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 5b5dc1190d203ff79365b9903c40512a, server=jenkins-hbase4.apache.org,39613,1700438000411 in 225 msec 2023-11-19 23:53:27,695 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/e30832324f8a4f02a1df76dba688043c.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5b5dc1190d203ff79365b9903c40512a, UNASSIGN in 239 msec 2023-11-19 23:53:27,697 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/03e59751fba34aedbbec7b142ac93238.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,698 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/515a0de064de4ff1b67559213cca8e40.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,699 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-7adc0a4e673a4221b6eae09d706cd16f 2023-11-19 23:53:27,701 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-6214522d85fe411aac06d486f0810868 2023-11-19 23:53:27,702 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-697ffcb6452c4c6eaf6b97a429c8c041 2023-11-19 23:53:27,704 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2] to archive 2023-11-19 23:53:27,705 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:27,707 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/676484a783d144a098ddf008fba19755.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,708 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/9bbd751b99334713960e8822136c5b1a.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,710 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/b7f89e290baa4633b27e8a86bb72fe36.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,711 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/6261b2d1f4a2425abb0082980775c56e.dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,712 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-cd4eef954cd74c90b8dc1df4119fab5a 2023-11-19 23:53:27,714 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-50879b95790f4a3d97148c1665b95021 2023-11-19 23:53:27,715 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/testSplitTableRegionUnevenDaughter=dae2bb8e46806b529da56f5c659c8d9b-0424de1a738546a98f618bf1559b23d2 2023-11-19 23:53:27,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:53:27,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:27,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3. 2023-11-19 23:53:27,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 31dbf9d0416d92b720428ba12f6265d3: 2023-11-19 23:53:27,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:27,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=31dbf9d0416d92b720428ba12f6265d3, regionState=CLOSED 2023-11-19 23:53:27,725 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1700438007724"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438007724"}]},"ts":"1700438007724"} 2023-11-19 23:53:27,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=22, resume processing ppid=21 2023-11-19 23:53:27,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 31dbf9d0416d92b720428ba12f6265d3, server=jenkins-hbase4.apache.org,39613,1700438000411 in 266 msec 2023-11-19 23:53:27,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=21, resume processing ppid=19 2023-11-19 23:53:27,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=31dbf9d0416d92b720428ba12f6265d3, UNASSIGN in 274 msec 2023-11-19 23:53:27,732 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438007732"}]},"ts":"1700438007732"} 2023-11-19 23:53:27,734 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-11-19 23:53:27,736 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-11-19 23:53:27,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 297 msec 2023-11-19 23:53:27,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-19 23:53:27,761 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-11-19 23:53:27,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,769 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:53:27,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,775 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,778 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-11-19 23:53:27,788 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,788 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,788 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:27,795 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/recovered.edits] 2023-11-19 23:53:27,796 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/recovered.edits] 2023-11-19 23:53:27,797 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/recovered.edits] 2023-11-19 23:53:27,801 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/7a4e03f73e0a4d4f94be3343cfa391b5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf1/7a4e03f73e0a4d4f94be3343cfa391b5 2023-11-19 23:53:27,802 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/03e59751fba34aedbbec7b142ac93238 2023-11-19 23:53:27,804 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/ed536f2d67ce463a966629b3e01e6e44 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf1/ed536f2d67ce463a966629b3e01e6e44 2023-11-19 23:53:27,806 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/515a0de064de4ff1b67559213cca8e40 2023-11-19 23:53:27,807 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/22948d3cec6b4c72b997ff3595353063 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/cf2/22948d3cec6b4c72b997ff3595353063 2023-11-19 23:53:27,810 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/6214522d85fe411aac06d486f0810868 2023-11-19 23:53:27,816 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/1fd316cac7e64a218592ae28a4e651f4 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/cf2/1fd316cac7e64a218592ae28a4e651f4 2023-11-19 23:53:27,818 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/697ffcb6452c4c6eaf6b97a429c8c041 2023-11-19 23:53:27,818 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3/recovered.edits/90.seqid 2023-11-19 23:53:27,819 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:27,821 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a/recovered.edits/90.seqid 2023-11-19 23:53:27,821 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/7adc0a4e673a4221b6eae09d706cd16f 2023-11-19 23:53:27,822 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/5b5dc1190d203ff79365b9903c40512a 2023-11-19 23:53:27,824 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/b685e341e30a4bd281afa6532d30102e 2023-11-19 23:53:27,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf1/e30832324f8a4f02a1df76dba688043c 2023-11-19 23:53:27,831 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/0424de1a738546a98f618bf1559b23d2 2023-11-19 23:53:27,835 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/50879b95790f4a3d97148c1665b95021 2023-11-19 23:53:27,837 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/6261b2d1f4a2425abb0082980775c56e 2023-11-19 23:53:27,838 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/676484a783d144a098ddf008fba19755 2023-11-19 23:53:27,840 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/9bbd751b99334713960e8822136c5b1a 2023-11-19 23:53:27,842 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/b7f89e290baa4633b27e8a86bb72fe36 2023-11-19 23:53:27,845 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/cf2/cd4eef954cd74c90b8dc1df4119fab5a 2023-11-19 23:53:27,849 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b/recovered.edits/85.seqid 2023-11-19 23:53:27,850 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionUnevenDaughter/dae2bb8e46806b529da56f5c659c8d9b 2023-11-19 23:53:27,850 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-11-19 23:53:27,853 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,855 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-11-19 23:53:27,855 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-11-19 23:53:27,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-11-19 23:53:27,884 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-11-19 23:53:27,926 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-11-19 23:53:27,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-11-19 23:53:27,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438007928"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:27,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438007928"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:27,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438007928"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:27,932 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:53:27,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => dae2bb8e46806b529da56f5c659c8d9b, NAME => 'testSplitTableRegionUnevenDaughter,,1700438003293.dae2bb8e46806b529da56f5c659c8d9b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5b5dc1190d203ff79365b9903c40512a, NAME => 'testSplitTableRegionUnevenDaughter,,1700438005466.5b5dc1190d203ff79365b9903c40512a.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 31dbf9d0416d92b720428ba12f6265d3, NAME => 'testSplitTableRegionUnevenDaughter,26,1700438005466.31dbf9d0416d92b720428ba12f6265d3.', STARTKEY => '26', ENDKEY => ''}] 2023-11-19 23:53:27,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-11-19 23:53:27,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438007932"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:27,934 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-11-19 23:53:27,937 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:27,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 171 msec 2023-11-19 23:53:28,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-11-19 23:53:28,084 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-11-19 23:53:28,102 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 360) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:46046 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4b9cda40-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:32858 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4b9cda40-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38298 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:33146 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38342 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:32906 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4b9cda40-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38390 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4b9cda40-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:46238 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38404 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:45962 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:39613-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:33184 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38306 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:46060 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:32840 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:45988 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38368 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=877 (was 723) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=331 (was 351), ProcessCount=172 (was 172), AvailableMemoryMB=15294 (was 15494) 2023-11-19 23:53:28,116 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=877, MaxFileDescriptor=60000, SystemLoadAverage=331, ProcessCount=172, AvailableMemoryMB=15293 2023-11-19 23:53:28,116 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:28,119 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:28,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:28,122 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:28,123 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:28,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:28,126 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,127 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d empty. 2023-11-19 23:53:28,127 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-11-19 23:53:28,139 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:28,140 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 515c2e69ac3f877374dbf5e5cd3d1f8d, NAME => 'testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 515c2e69ac3f877374dbf5e5cd3d1f8d, disabling compactions & flushes 2023-11-19 23:53:28,153 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. after waiting 0 ms 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,153 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,153 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:28,155 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:28,157 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438008156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438008156"}]},"ts":"1700438008156"} 2023-11-19 23:53:28,159 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:28,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:28,160 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438008160"}]},"ts":"1700438008160"} 2023-11-19 23:53:28,161 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-11-19 23:53:28,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:28,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:28,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:28,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:28,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:28,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, ASSIGN}] 2023-11-19 23:53:28,169 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, ASSIGN 2023-11-19 23:53:28,169 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:53:28,320 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:28,321 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,321 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438008321"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438008321"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438008321"}]},"ts":"1700438008321"} 2023-11-19 23:53:28,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:28,476 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,477 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-19 23:53:28,480 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48842, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-19 23:53:28,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 515c2e69ac3f877374dbf5e5cd3d1f8d, NAME => 'testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:28,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:28,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:28,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,485 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,486 DEBUG [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1 2023-11-19 23:53:28,487 DEBUG [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1 2023-11-19 23:53:28,487 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 515c2e69ac3f877374dbf5e5cd3d1f8d columnFamilyName cf1 2023-11-19 23:53:28,488 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] regionserver.HStore(310): Store=515c2e69ac3f877374dbf5e5cd3d1f8d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:28,488 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,489 DEBUG [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2 2023-11-19 23:53:28,489 DEBUG [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2 2023-11-19 23:53:28,491 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 515c2e69ac3f877374dbf5e5cd3d1f8d columnFamilyName cf2 2023-11-19 23:53:28,491 INFO [StoreOpener-515c2e69ac3f877374dbf5e5cd3d1f8d-1] regionserver.HStore(310): Store=515c2e69ac3f877374dbf5e5cd3d1f8d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:28,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:28,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:28,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:28,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 515c2e69ac3f877374dbf5e5cd3d1f8d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65336398, jitterRate=-0.02641180157661438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:28,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:28,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d., pid=27, masterSystemTime=1700438008476 2023-11-19 23:53:28,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,506 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,506 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438008505"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438008505"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438008505"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438008505"}]},"ts":"1700438008505"} 2023-11-19 23:53:28,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=27, resume processing ppid=26 2023-11-19 23:53:28,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 in 184 msec 2023-11-19 23:53:28,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=26, resume processing ppid=25 2023-11-19 23:53:28,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, ASSIGN in 343 msec 2023-11-19 23:53:28,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:28,513 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438008513"}]},"ts":"1700438008513"} 2023-11-19 23:53:28,515 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-11-19 23:53:28,518 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:28,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 398 msec 2023-11-19 23:53:28,625 DEBUG [Listener at localhost/42385] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-19 23:53:28,629 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48854, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-19 23:53:28,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:28,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:28,642 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:28,642 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:28,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:28,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,646 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,646 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,646 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,646 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,646 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,646 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:28,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,648 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,648 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,648 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,649 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,649 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:28,653 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:28,652 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,653 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,653 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:28,653 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:53:28,655 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,655 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,655 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,656 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,656 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,656 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,667 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/ca7baad3fad14bd491feda5bb5ccd9ba 2023-11-19 23:53:28,681 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/74a78c611df34bffa7a4ee5ef7baa220 2023-11-19 23:53:28,686 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/ca7baad3fad14bd491feda5bb5ccd9ba as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba 2023-11-19 23:53:28,691 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:28,692 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/74a78c611df34bffa7a4ee5ef7baa220 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 2023-11-19 23:53:28,697 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:28,698 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 45ms, sequenceid=6, compaction requested=false 2023-11-19 23:53:28,698 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-11-19 23:53:28,698 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:28,699 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:28,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:28,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,701 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,701 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,701 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,701 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:28,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@179e7e6a[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:28,703 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:28,704 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,705 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,705 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,705 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,706 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,706 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:28,706 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,708 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,709 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,709 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,710 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,708 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:28,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:28,715 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:28,715 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:28,715 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:28,715 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:28,715 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,718 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,717 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,718 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,719 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,721 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,721 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,722 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,729 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:28,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:28,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,731 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,831 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:28,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:28,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:28,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:28,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:28,851 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:28,852 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,852 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:28,853 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:28,853 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,853 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,857 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,857 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:28,858 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,858 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:28,859 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,859 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:28,859 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:28,859 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:28,860 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,860 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:28,860 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:28,861 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,861 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,861 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,862 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,862 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,862 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,871 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/9a548173596645a294d91315a19376ad 2023-11-19 23:53:28,891 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 2023-11-19 23:53:28,897 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/9a548173596645a294d91315a19376ad as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad 2023-11-19 23:53:28,902 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:28,903 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 2023-11-19 23:53:28,908 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:28,909 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 49ms, sequenceid=19, compaction requested=false 2023-11-19 23:53:28,910 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:28,910 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:28,914 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,914 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,914 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,914 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:28,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@551985b[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:28,916 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:28,916 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,918 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:28,920 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,920 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,920 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,920 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,921 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:28,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:28,929 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:28,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:28,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:28,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:28,930 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,933 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:28,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,934 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:28,934 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:28,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,935 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:28,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,938 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,938 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,938 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,942 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,942 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,942 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,943 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:28,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:28,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:28,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:28,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:28,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:28,944 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:28,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:28,944 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:28,944 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:28,944 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:28,945 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:28,945 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:28,945 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,044 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:29,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:29,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:29,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:29,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:29,070 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:29,070 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,071 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:29,071 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:29,072 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,072 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,073 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,073 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,074 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,073 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,074 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,074 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,076 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,076 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,076 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,076 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,081 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,081 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,082 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:29,082 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,082 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,082 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,082 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:29,084 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,084 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,084 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,085 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,086 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,086 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,100 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/134b1ae9e3334a6f921dd85f3c84b89b 2023-11-19 23:53:29,115 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/7252065028b64ab2862e44e1e62ae969 2023-11-19 23:53:29,121 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/134b1ae9e3334a6f921dd85f3c84b89b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b 2023-11-19 23:53:29,126 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:29,127 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/7252065028b64ab2862e44e1e62ae969 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 2023-11-19 23:53:29,131 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:29,132 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 50ms, sequenceid=32, compaction requested=true 2023-11-19 23:53:29,132 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:29,132 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,132 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:29,132 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:29,132 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,132 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,132 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,134 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,134 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,134 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,134 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:29,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20a6425e[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:29,137 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:29,137 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,138 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,139 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:29,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,139 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,139 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:29,145 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,146 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,146 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,147 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,147 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,148 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-19 23:53:29,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,148 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-11-19 23:53:29,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,149 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:29,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,149 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-11-19 23:53:29,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:29,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:29,152 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:29,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:29,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:29,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:29,152 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,155 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,156 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,156 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,160 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,161 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,161 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,165 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,165 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,165 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,166 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:29,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:29,166 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,167 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,267 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:29,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:29,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:29,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:29,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:29,290 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:29,290 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,291 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:29,291 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:29,292 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,292 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,293 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,293 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,293 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,293 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,294 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,294 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,296 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,296 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,297 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,298 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,298 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:29,298 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:29,298 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,298 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,299 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:29,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,300 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:29,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,303 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,303 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,311 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/4697d2838b3f4a63af7d398839efaf0c 2023-11-19 23:53:29,326 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/46a48d4d6953468dae04e436de09137a 2023-11-19 23:53:29,331 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/4697d2838b3f4a63af7d398839efaf0c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c 2023-11-19 23:53:29,336 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:29,336 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/46a48d4d6953468dae04e436de09137a as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a 2023-11-19 23:53:29,342 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:29,343 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 44ms, sequenceid=45, compaction requested=true 2023-11-19 23:53:29,343 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:29,343 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:29,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:29,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,345 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,345 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,345 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,345 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:29,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@763a6b73[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:29,348 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:29,348 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,350 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:29,350 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:29,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,350 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,352 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,353 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,353 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,353 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:29,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:29,357 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:29,357 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:29,357 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:29,357 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:29,357 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,360 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,359 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,360 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,361 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,364 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,365 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,365 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,370 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:29,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,370 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,371 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:29,371 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,371 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,371 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,371 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,372 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,372 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,471 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:29,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:29,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:29,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:29,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:29,493 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:29,493 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,493 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:29,493 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:29,494 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,494 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,496 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,496 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,497 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,497 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,498 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,498 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,498 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,498 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:29,499 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,499 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:29,499 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:29,500 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,500 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,510 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/5ff559b5d51e4608b289b7eab3b17348 2023-11-19 23:53:29,525 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/f428257df22a49a093cee1d6e796f2ac 2023-11-19 23:53:29,530 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/5ff559b5d51e4608b289b7eab3b17348 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 2023-11-19 23:53:29,534 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:29,535 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/f428257df22a49a093cee1d6e796f2ac as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac 2023-11-19 23:53:29,539 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:29,540 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 41ms, sequenceid=58, compaction requested=true 2023-11-19 23:53:29,540 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:29,540 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,540 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:29,541 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:29,541 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,541 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,541 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,542 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,542 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,543 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,543 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:29,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@123326f0[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:29,545 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:29,545 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,546 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:29,546 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,547 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:29,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:29,547 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:29,549 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,549 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,549 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,551 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,551 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:29,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:29,555 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:29,555 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:29,555 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:29,555 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:29,555 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,558 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,558 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,559 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,561 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,561 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,562 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,565 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:29,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,566 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,565 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,566 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:29,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,566 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,566 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:29,567 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,567 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,567 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,567 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,567 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,667 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:29,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:29,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:29,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:29,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-11-19 23:53:29,689 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-11-19 23:53:29,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-11-19 23:53:29,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-11-19 23:53:29,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,692 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,692 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,692 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-11-19 23:53:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,695 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,695 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:29,695 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:29,695 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,695 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. started... 2023-11-19 23:53:29,695 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:29,696 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,696 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,696 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,697 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,697 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,698 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,705 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/9f5411b9411d4101bb7954c6790f4a95 2023-11-19 23:53:29,719 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/0dc0cd2cf9814698b9ba6a4496141d16 2023-11-19 23:53:29,724 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/9f5411b9411d4101bb7954c6790f4a95 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 2023-11-19 23:53:29,730 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:29,730 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/0dc0cd2cf9814698b9ba6a4496141d16 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 2023-11-19 23:53:29,736 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:29,736 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 41ms, sequenceid=71, compaction requested=true 2023-11-19 23:53:29,737 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:29,737 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-11-19 23:53:29,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-11-19 23:53:29,738 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,738 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,739 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,739 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:29,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-11-19 23:53:29,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ba9f7ad[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:29,741 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-11-19 23:53:29,742 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,744 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,744 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:29,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:29,744 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-11-19 23:53:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-11-19 23:53:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,749 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-11-19 23:53:29,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,749 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-11-19 23:53:29,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:53:29,753 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-11-19 23:53:29,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:29,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:29,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-11-19 23:53:29,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-11-19 23:53:29,754 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,756 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,756 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,756 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:29,757 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:29,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-11-19 23:53:29,760 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-11-19 23:53:29,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-11-19 23:53:29,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-11-19 23:53:29,764 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-11-19 23:53:29,864 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:29,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:30,129 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-11-19 23:53:30,130 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:30,131 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,132 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,132 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,132 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,133 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:30,133 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:30,133 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:30,381 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,382 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:30,382 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:30,383 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:30,385 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-11-19 23:53:30,386 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:30,386 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:30,386 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:30,386 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:30,387 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:30,389 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-11-19 23:53:30,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:30,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:30,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:30,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:30,392 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,392 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:30,392 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,393 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,393 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:30,394 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,394 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:30,394 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,394 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,394 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,394 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:53:30,395 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:30,396 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:30,398 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438040398 2023-11-19 23:53:30,399 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:30,399 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,403 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:30,403 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:30,403 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:30,403 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438070403 2023-11-19 23:53:30,403 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438130403 2023-11-19 23:53:30,404 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:30,404 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:30,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:30,406 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:30,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:30,654 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:30,654 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:30,655 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:30,655 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:30,657 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-11-19 23:53:30,657 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:30,658 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:30,658 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:30,658 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:30,659 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,659 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:30,659 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,659 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,660 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:30,660 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,660 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:30,660 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,660 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,660 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,660 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:30,661 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:30,662 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:30,662 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438040662 2023-11-19 23:53:30,663 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:30,663 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,666 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:30,666 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:30,667 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:30,667 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438070667 2023-11-19 23:53:30,667 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438130667 2023-11-19 23:53:30,667 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:30,667 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:30,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN}] 2023-11-19 23:53:30,669 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:30,669 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:30,669 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:30,917 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,917 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:30,917 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:30,917 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:30,917 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:30,918 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:30,918 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:30,918 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:30,918 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:30,918 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:30,920 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-11-19 23:53:30,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:30,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:30,922 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:30,922 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:30,923 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,923 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:30,923 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,924 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:30,924 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:30,925 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,925 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:30,925 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,925 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,925 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,925 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:30,926 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:30,926 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:30,927 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438040927 2023-11-19 23:53:30,928 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:30,928 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:30,931 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:30,932 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:30,932 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:30,932 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438070932 2023-11-19 23:53:30,932 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438130932 2023-11-19 23:53:30,932 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:30,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN}] 2023-11-19 23:53:30,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:30,934 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:30,935 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:30,935 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438010935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438010935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438010935"}]},"ts":"1700438010935"} 2023-11-19 23:53:30,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:30,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:30,937 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:30,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:30,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:31,182 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,182 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:31,182 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:31,183 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:31,183 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:31,183 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:31,183 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:31,183 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:31,183 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:31,184 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:31,185 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:31,186 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:31,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:31,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:31,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:31,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:31,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:31,188 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,188 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,188 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:31,189 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,189 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,189 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,189 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,189 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,190 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,190 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,190 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,190 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:31,190 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:31,191 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:31,191 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d to restore RIT 2023-11-19 23:53:31,191 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438041191 2023-11-19 23:53:31,192 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:31,192 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,194 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:31,195 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:31,195 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:31,195 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438071195 2023-11-19 23:53:31,195 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438131195 2023-11-19 23:53:31,195 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:53:31,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438011195"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438011195"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438011195"}]},"ts":"1700438011195"} 2023-11-19 23:53:31,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:31,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:31,350 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:31,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 515c2e69ac3f877374dbf5e5cd3d1f8d, disabling compactions & flushes 2023-11-19 23:53:31,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:31,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:31,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. after waiting 0 ms 2023-11-19 23:53:31,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:31,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 515c2e69ac3f877374dbf5e5cd3d1f8d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:53:31,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/36549872970243b299a8a2d3377ba3a5 2023-11-19 23:53:31,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/0425965d6b3543f585f733e0e233de1b 2023-11-19 23:53:31,384 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf1/36549872970243b299a8a2d3377ba3a5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 2023-11-19 23:53:31,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:31,390 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/.tmp/cf2/0425965d6b3543f585f733e0e233de1b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b 2023-11-19 23:53:31,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:31,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 515c2e69ac3f877374dbf5e5cd3d1f8d in 45ms, sequenceid=82, compaction requested=true 2023-11-19 23:53:31,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:53:31,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:31,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. 2023-11-19 23:53:31,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 515c2e69ac3f877374dbf5e5cd3d1f8d: 2023-11-19 23:53:31,420 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:31,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED 2023-11-19 23:53:31,421 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438011420"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438011420"}]},"ts":"1700438011420"} 2023-11-19 23:53:31,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:31,422 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:31,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:31,445 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,445 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:31,445 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:31,445 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:31,446 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:31,447 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:31,447 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:31,447 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:31,447 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:31,448 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:31,450 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:31,450 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-11-19 23:53:31,450 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-11-19 23:53:31,450 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:31,451 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:31,451 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:31,451 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:31,452 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,452 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:31,453 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,453 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,453 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:31,454 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,454 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,454 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,455 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,455 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,455 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,455 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,455 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,455 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,455 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,456 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:53:31,456 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:31,457 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:31,457 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d to restore RIT 2023-11-19 23:53:31,458 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438041458 2023-11-19 23:53:31,458 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,458 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:31,461 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:31,461 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:31,462 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:31,462 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438071462 2023-11-19 23:53:31,462 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438131462 2023-11-19 23:53:31,462 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:31,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438011462"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438011462"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438011462"}]},"ts":"1700438011462"} 2023-11-19 23:53:31,465 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:31,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=30, resume processing ppid=29 2023-11-19 23:53:31,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 in 268 msec 2023-11-19 23:53:31,468 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:31,468 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:31,468 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:31,712 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,712 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:31,712 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:31,712 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:31,713 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:31,713 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:31,713 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:31,713 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:31,713 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:31,714 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:31,715 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:31,715 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-11-19 23:53:31,715 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:31,715 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:31,716 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:31,716 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:31,717 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:31,718 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,718 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:31,718 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,718 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,719 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:31,719 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,719 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,720 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,720 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:31,720 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,720 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,720 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,720 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:31,720 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:31,720 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:31,721 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:31,721 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:31,721 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d to restore RIT 2023-11-19 23:53:31,722 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438041722 2023-11-19 23:53:31,723 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:31,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,725 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:31,726 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:31,726 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:31,726 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438071726 2023-11-19 23:53:31,726 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438131726 2023-11-19 23:53:31,726 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:31,727 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:31,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=29, resume processing ppid=28 2023-11-19 23:53:31,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN in 797 msec 2023-11-19 23:53:31,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:31,742 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:31,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:31,977 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,977 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:31,977 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:31,977 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:31,978 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:31,978 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:31,978 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:31,978 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:31,979 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:31,979 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:31,981 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:31,981 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:31,981 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:31,981 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:31,981 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:31,982 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:31,982 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:31,982 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,982 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:31,983 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,983 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:31,983 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:31,984 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:31,984 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:31,985 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:31,985 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:31,985 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438041985 2023-11-19 23:53:31,987 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:31,987 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,990 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:31,990 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:31,990 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:31,990 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438071990 2023-11-19 23:53:31,990 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438131990 2023-11-19 23:53:31,991 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:31,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:31,996 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:32,007 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=515c2e69ac3f877374dbf5e5cd3d1f8d, threads=14 2023-11-19 23:53:32,008 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,008 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,008 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,009 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,010 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,010 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,010 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,010 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,011 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,144 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,179 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,180 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,187 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,194 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,198 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,199 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,204 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,204 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,208 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,208 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,209 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,209 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,216 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,216 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 515c2e69ac3f877374dbf5e5cd3d1f8d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:32,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:32,258 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:32,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:32,491 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,491 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:32,491 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:32,491 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:32,493 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:32,493 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:32,493 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:32,493 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:32,493 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:32,494 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:32,496 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-11-19 23:53:32,496 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:32,496 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:32,497 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:32,497 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:32,497 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:32,498 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:32,498 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:32,499 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:32,499 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:32,500 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:32,500 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:32,501 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,501 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,501 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:32,501 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:32,501 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:53:32,502 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:32,503 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:32,504 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438042504 2023-11-19 23:53:32,506 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,506 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:32,509 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:32,509 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:32,509 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:32,509 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438072509 2023-11-19 23:53:32,509 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438132509 2023-11-19 23:53:32,509 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:32,528 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=515c2e69ac3f877374dbf5e5cd3d1f8d, threads=14 2023-11-19 23:53:32,532 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,536 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,540 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,543 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,557 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,560 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,563 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,564 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,566 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,571 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,577 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,580 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,581 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,584 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,774 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,801 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,804 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,836 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,840 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,842 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,848 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,850 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,851 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,860 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,861 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,864 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,865 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,870 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac for region: 515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:32,871 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 515c2e69ac3f877374dbf5e5cd3d1f8d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:32,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:32,929 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:32,931 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:32,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:32,931 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:32,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:32,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:33,010 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:33,010 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:33,011 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:33,011 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:33,012 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-11-19 23:53:33,013 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:33,013 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:33,013 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:33,014 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:33,014 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:33,014 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:33,015 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,015 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:33,015 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,016 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,016 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:33,017 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,017 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:33,017 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,017 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:33,018 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,018 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,018 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,018 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:33,018 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:33,018 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:33,019 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:33,019 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:33,020 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438043020 2023-11-19 23:53:33,021 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:33,021 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,023 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:33,024 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:33,024 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:33,024 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438073024 2023-11-19 23:53:33,024 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438133024 2023-11-19 23:53:33,024 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:33,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:33,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:33,029 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:33,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:33,186 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:53:33,274 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:33,275 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:33,275 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:33,275 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:33,276 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:33,276 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:33,276 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:33,276 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:33,277 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:33,277 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:33,279 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-11-19 23:53:33,279 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:33,279 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:33,279 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:33,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:33,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:33,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:33,281 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,281 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:33,282 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,282 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:33,282 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:33,283 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,283 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN 2023-11-19 23:53:33,283 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,283 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:33,284 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,284 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,284 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,284 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=515c2e69ac3f877374dbf5e5cd3d1f8d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:33,284 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 515c2e69ac3f877374dbf5e5cd3d1f8d, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:33,284 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:33,285 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:33,286 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:33,292 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438043292 2023-11-19 23:53:33,293 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:33,294 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,297 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:33,297 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=515c2e69ac3f877374dbf5e5cd3d1f8d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:53:33,297 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:33,297 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438073297 2023-11-19 23:53:33,297 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438133297 2023-11-19 23:53:33,298 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:33,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438013301"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1700438013301"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1700438013301"}]},"ts":"1700438013301"} 2023-11-19 23:53:33,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013301"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438013301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438013301"}]},"ts":"1700438013301"} 2023-11-19 23:53:33,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013301"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438013301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438013301"}]},"ts":"1700438013301"} 2023-11-19 23:53:33,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, ASSIGN}] 2023-11-19 23:53:33,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, ASSIGN 2023-11-19 23:53:33,316 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:53:33,317 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, ASSIGN 2023-11-19 23:53:33,318 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:53:33,466 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:53:33,468 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0c4a4342de5afa19f4fa73194454b024, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,468 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013468"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438013468"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438013468"}]},"ts":"1700438013468"} 2023-11-19 23:53:33,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 0c4a4342de5afa19f4fa73194454b024, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:33,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0b50a5610148ac95f35e1e3c642779e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438013473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438013473"}]},"ts":"1700438013473"} 2023-11-19 23:53:33,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 0b50a5610148ac95f35e1e3c642779e7, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:33,622 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0c4a4342de5afa19f4fa73194454b024, NAME => 'testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.', STARTKEY => '', ENDKEY => '41'} 2023-11-19 23:53:33,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:33,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:33,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,628 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,629 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1 2023-11-19 23:53:33,629 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1 2023-11-19 23:53:33,629 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c4a4342de5afa19f4fa73194454b024 columnFamilyName cf1 2023-11-19 23:53:33,645 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-bottom 2023-11-19 23:53:33,650 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-bottom 2023-11-19 23:53:33,655 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-bottom 2023-11-19 23:53:33,660 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-bottom 2023-11-19 23:53:33,666 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-bottom 2023-11-19 23:53:33,671 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-bottom 2023-11-19 23:53:33,676 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-bottom 2023-11-19 23:53:33,677 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(310): Store=0c4a4342de5afa19f4fa73194454b024/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:33,677 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,678 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2 2023-11-19 23:53:33,678 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2 2023-11-19 23:53:33,678 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c4a4342de5afa19f4fa73194454b024 columnFamilyName cf2 2023-11-19 23:53:33,693 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-bottom 2023-11-19 23:53:33,699 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-bottom 2023-11-19 23:53:33,705 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-bottom 2023-11-19 23:53:33,709 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-bottom 2023-11-19 23:53:33,714 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-bottom 2023-11-19 23:53:33,721 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-bottom 2023-11-19 23:53:33,731 DEBUG [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-bottom 2023-11-19 23:53:33,732 INFO [StoreOpener-0c4a4342de5afa19f4fa73194454b024-1] regionserver.HStore(310): Store=0c4a4342de5afa19f4fa73194454b024/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:33,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:33,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:33,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0c4a4342de5afa19f4fa73194454b024; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64105851, jitterRate=-0.04474838078022003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:33,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0c4a4342de5afa19f4fa73194454b024: 2023-11-19 23:53:33,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024., pid=33, masterSystemTime=1700438013622 2023-11-19 23:53:33,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:33,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:33,741 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:33,741 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:33,741 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,741 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): 0c4a4342de5afa19f4fa73194454b024/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:33,741 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,741 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c4a4342de5afa19f4fa73194454b024/cf2 in testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,741 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): 0c4a4342de5afa19f4fa73194454b024/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:33,741 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0c4a4342de5afa19f4fa73194454b024/cf1 in testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,741 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/.tmp, totalSize=34.9 K 2023-11-19 23:53:33,742 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438008630 2023-11-19 23:53:33,742 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/.tmp, totalSize=34.9 K 2023-11-19 23:53:33,743 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438008833 2023-11-19 23:53:33,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,743 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438008630 2023-11-19 23:53:33,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:33,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0b50a5610148ac95f35e1e3c642779e7, NAME => 'testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.', STARTKEY => '41', ENDKEY => ''} 2023-11-19 23:53:33,743 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438009047 2023-11-19 23:53:33,743 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=0c4a4342de5afa19f4fa73194454b024, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:33,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013743"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438013743"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438013743"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438013743"}]},"ts":"1700438013743"} 2023-11-19 23:53:33,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:33,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,744 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438009270 2023-11-19 23:53:33,745 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438008833 2023-11-19 23:53:33,745 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438009474 2023-11-19 23:53:33,746 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438009669 2023-11-19 23:53:33,747 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438009866 2023-11-19 23:53:33,748 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438009047 2023-11-19 23:53:33,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=33, resume processing ppid=31 2023-11-19 23:53:33,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 0c4a4342de5afa19f4fa73194454b024, server=jenkins-hbase4.apache.org,36283,1700438000319 in 276 msec 2023-11-19 23:53:33,749 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438009270 2023-11-19 23:53:33,750 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,751 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438009474 2023-11-19 23:53:33,752 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438009669 2023-11-19 23:53:33,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, ASSIGN in 437 msec 2023-11-19 23:53:33,754 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1 2023-11-19 23:53:33,754 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1 2023-11-19 23:53:33,755 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0b50a5610148ac95f35e1e3c642779e7 columnFamilyName cf1 2023-11-19 23:53:33,755 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438009866 2023-11-19 23:53:33,779 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): 0c4a4342de5afa19f4fa73194454b024#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:33,796 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0c4a4342de5afa19f4fa73194454b024#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:33,800 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-top 2023-11-19 23:53:33,805 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-top 2023-11-19 23:53:33,809 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-top 2023-11-19 23:53:33,814 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-top 2023-11-19 23:53:33,819 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-top 2023-11-19 23:53:33,828 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-top 2023-11-19 23:53:33,831 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/.tmp/cf2/f37f2291490541778b33bd1a08d62fd1 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f37f2291490541778b33bd1a08d62fd1 2023-11-19 23:53:33,842 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c4a4342de5afa19f4fa73194454b024/cf2 of 0c4a4342de5afa19f4fa73194454b024 into f37f2291490541778b33bd1a08d62fd1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:33,842 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c4a4342de5afa19f4fa73194454b024: 2023-11-19 23:53:33,842 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024., storeName=0c4a4342de5afa19f4fa73194454b024/cf2, priority=9, startTime=1700438013740; duration=0sec 2023-11-19 23:53:33,843 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:33,863 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-top 2023-11-19 23:53:33,864 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(310): Store=0b50a5610148ac95f35e1e3c642779e7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:33,864 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,865 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2 2023-11-19 23:53:33,865 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2 2023-11-19 23:53:33,866 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0b50a5610148ac95f35e1e3c642779e7 columnFamilyName cf2 2023-11-19 23:53:33,878 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/.tmp/cf1/9832be1b663d4774ac3a2edf5626ec08 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9832be1b663d4774ac3a2edf5626ec08 2023-11-19 23:53:33,886 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0c4a4342de5afa19f4fa73194454b024/cf1 of 0c4a4342de5afa19f4fa73194454b024 into 9832be1b663d4774ac3a2edf5626ec08(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:33,886 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0c4a4342de5afa19f4fa73194454b024: 2023-11-19 23:53:33,886 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024., storeName=0c4a4342de5afa19f4fa73194454b024/cf1, priority=9, startTime=1700438013739; duration=0sec 2023-11-19 23:53:33,887 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:33,888 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-top 2023-11-19 23:53:33,893 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-top 2023-11-19 23:53:33,901 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-top 2023-11-19 23:53:33,905 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-top 2023-11-19 23:53:33,909 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-top 2023-11-19 23:53:33,914 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-top 2023-11-19 23:53:33,918 DEBUG [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-top 2023-11-19 23:53:33,918 INFO [StoreOpener-0b50a5610148ac95f35e1e3c642779e7-1] regionserver.HStore(310): Store=0b50a5610148ac95f35e1e3c642779e7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:33,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:33,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:33,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0b50a5610148ac95f35e1e3c642779e7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63723871, jitterRate=-0.05044032633304596}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:33,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0b50a5610148ac95f35e1e3c642779e7: 2023-11-19 23:53:33,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7., pid=34, masterSystemTime=1700438013622 2023-11-19 23:53:33,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:33,929 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:33,929 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:33,929 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:33,929 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): 0b50a5610148ac95f35e1e3c642779e7/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:33,929 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0b50a5610148ac95f35e1e3c642779e7/cf1 in testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,930 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/.tmp, totalSize=34.9 K 2023-11-19 23:53:33,929 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,930 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): 0b50a5610148ac95f35e1e3c642779e7/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:33,930 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0b50a5610148ac95f35e1e3c642779e7/cf2 in testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,931 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/.tmp, totalSize=34.9 K 2023-11-19 23:53:33,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:33,932 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438008630 2023-11-19 23:53:33,932 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0b50a5610148ac95f35e1e3c642779e7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:33,933 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438013932"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438013932"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438013932"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438013932"}]},"ts":"1700438013932"} 2023-11-19 23:53:33,933 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438008833 2023-11-19 23:53:33,932 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438008630 2023-11-19 23:53:33,934 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438009047 2023-11-19 23:53:33,935 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438009270 2023-11-19 23:53:33,935 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438008833 2023-11-19 23:53:33,935 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438009474 2023-11-19 23:53:33,936 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438009047 2023-11-19 23:53:33,937 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438009270 2023-11-19 23:53:33,937 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438009669 2023-11-19 23:53:33,937 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438009474 2023-11-19 23:53:33,938 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438009866 2023-11-19 23:53:33,938 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438009669 2023-11-19 23:53:33,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=34, resume processing ppid=32 2023-11-19 23:53:33,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 0b50a5610148ac95f35e1e3c642779e7, server=jenkins-hbase4.apache.org,36283,1700438000319 in 461 msec 2023-11-19 23:53:33,940 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438009866 2023-11-19 23:53:33,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=32, resume processing ppid=28 2023-11-19 23:53:33,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, ASSIGN in 628 msec 2023-11-19 23:53:33,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 in 3.8130 sec 2023-11-19 23:53:33,973 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): 0b50a5610148ac95f35e1e3c642779e7#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:33,976 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0b50a5610148ac95f35e1e3c642779e7#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:34,011 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/.tmp/cf2/59abfe8fdba5424c966fd94a3f60802d as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/59abfe8fdba5424c966fd94a3f60802d 2023-11-19 23:53:34,015 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/.tmp/cf1/320b38d58d194c54b2b9b96d9fbbef83 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/320b38d58d194c54b2b9b96d9fbbef83 2023-11-19 23:53:34,019 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0b50a5610148ac95f35e1e3c642779e7/cf2 of 0b50a5610148ac95f35e1e3c642779e7 into 59abfe8fdba5424c966fd94a3f60802d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:34,020 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0b50a5610148ac95f35e1e3c642779e7: 2023-11-19 23:53:34,020 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7., storeName=0b50a5610148ac95f35e1e3c642779e7/cf2, priority=9, startTime=1700438013929; duration=0sec 2023-11-19 23:53:34,020 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:34,022 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0b50a5610148ac95f35e1e3c642779e7/cf1 of 0b50a5610148ac95f35e1e3c642779e7 into 320b38d58d194c54b2b9b96d9fbbef83(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:34,022 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0b50a5610148ac95f35e1e3c642779e7: 2023-11-19 23:53:34,022 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7., storeName=0b50a5610148ac95f35e1e3c642779e7/cf1, priority=9, startTime=1700438013927; duration=0sec 2023-11-19 23:53:34,022 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:34,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36283] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:48854 deadline: 1700438074049, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d. is not online on jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:34,162 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:53:34,173 INFO [Listener at localhost/42385] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-11-19 23:53:34,175 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:53:34,197 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:34,198 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-11-19 23:53:34,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-11-19 23:53:34,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-19 23:53:34,202 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438014202"}]},"ts":"1700438014202"} 2023-11-19 23:53:34,204 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-11-19 23:53:34,206 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-11-19 23:53:34,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, UNASSIGN}] 2023-11-19 23:53:34,208 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, UNASSIGN 2023-11-19 23:53:34,208 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0b50a5610148ac95f35e1e3c642779e7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:34,208 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438014208"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438014208"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438014208"}]},"ts":"1700438014208"} 2023-11-19 23:53:34,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 0b50a5610148ac95f35e1e3c642779e7, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:34,212 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, UNASSIGN 2023-11-19 23:53:34,212 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0c4a4342de5afa19f4fa73194454b024, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:34,213 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438014212"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438014212"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438014212"}]},"ts":"1700438014212"} 2023-11-19 23:53:34,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 0c4a4342de5afa19f4fa73194454b024, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:53:34,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-19 23:53:34,361 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:34,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:34,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0b50a5610148ac95f35e1e3c642779e7, disabling compactions & flushes 2023-11-19 23:53:34,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:34,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:34,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. after waiting 0 ms 2023-11-19 23:53:34,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:34,373 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-top] to archive 2023-11-19 23:53:34,374 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:34,376 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,377 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,378 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,379 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,381 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,382 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,383 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,385 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-top] to archive 2023-11-19 23:53:34,385 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:34,387 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,389 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,390 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,391 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,392 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,393 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,394 DEBUG [StoreCloser-testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:53:34,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:34,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7. 2023-11-19 23:53:34,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0b50a5610148ac95f35e1e3c642779e7: 2023-11-19 23:53:34,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:34,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:34,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0c4a4342de5afa19f4fa73194454b024, disabling compactions & flushes 2023-11-19 23:53:34,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:34,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:34,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. after waiting 0 ms 2023-11-19 23:53:34,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:34,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0b50a5610148ac95f35e1e3c642779e7, regionState=CLOSED 2023-11-19 23:53:34,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438014401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438014401"}]},"ts":"1700438014401"} 2023-11-19 23:53:34,401 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5-bottom] to archive 2023-11-19 23:53:34,402 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:34,404 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=38, resume processing ppid=37 2023-11-19 23:53:34,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 0b50a5610148ac95f35e1e3c642779e7, server=jenkins-hbase4.apache.org,36283,1700438000319 in 192 msec 2023-11-19 23:53:34,406 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0b50a5610148ac95f35e1e3c642779e7, UNASSIGN in 198 msec 2023-11-19 23:53:34,407 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,409 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,410 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,411 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,412 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,414 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b-bottom] to archive 2023-11-19 23:53:34,415 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:34,416 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,417 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,418 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,419 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,420 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,421 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,422 DEBUG [StoreCloser-testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:53:34,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:34,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024. 2023-11-19 23:53:34,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0c4a4342de5afa19f4fa73194454b024: 2023-11-19 23:53:34,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:34,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=0c4a4342de5afa19f4fa73194454b024, regionState=CLOSED 2023-11-19 23:53:34,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438014429"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438014429"}]},"ts":"1700438014429"} 2023-11-19 23:53:34,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=39, resume processing ppid=36 2023-11-19 23:53:34,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 0c4a4342de5afa19f4fa73194454b024, server=jenkins-hbase4.apache.org,36283,1700438000319 in 216 msec 2023-11-19 23:53:34,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=36, resume processing ppid=35 2023-11-19 23:53:34,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0c4a4342de5afa19f4fa73194454b024, UNASSIGN in 225 msec 2023-11-19 23:53:34,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438014435"}]},"ts":"1700438014435"} 2023-11-19 23:53:34,436 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-11-19 23:53:34,438 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-11-19 23:53:34,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 240 msec 2023-11-19 23:53:34,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-19 23:53:34,504 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-11-19 23:53:34,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-11-19 23:53:34,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,508 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,508 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-11-19 23:53:34,516 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:34,516 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:34,516 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,518 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/recovered.edits] 2023-11-19 23:53:34,518 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/recovered.edits] 2023-11-19 23:53:34,518 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/recovered.edits] 2023-11-19 23:53:34,522 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/.regioninfo to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/.regioninfo 2023-11-19 23:53:34,523 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/.regioninfo to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/.regioninfo 2023-11-19 23:53:34,523 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/134b1ae9e3334a6f921dd85f3c84b89b 2023-11-19 23:53:34,525 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/36549872970243b299a8a2d3377ba3a5 2023-11-19 23:53:34,527 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,527 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/134b1ae9e3334a6f921dd85f3c84b89b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,528 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/4697d2838b3f4a63af7d398839efaf0c 2023-11-19 23:53:34,528 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,529 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/36549872970243b299a8a2d3377ba3a5.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,529 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/5ff559b5d51e4608b289b7eab3b17348 2023-11-19 23:53:34,530 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,531 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/4697d2838b3f4a63af7d398839efaf0c.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,531 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9a548173596645a294d91315a19376ad 2023-11-19 23:53:34,532 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,533 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/5ff559b5d51e4608b289b7eab3b17348.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,533 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/9f5411b9411d4101bb7954c6790f4a95 2023-11-19 23:53:34,533 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,535 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/9a548173596645a294d91315a19376ad.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,535 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf1/ca7baad3fad14bd491feda5bb5ccd9ba 2023-11-19 23:53:34,535 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,537 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/9f5411b9411d4101bb7954c6790f4a95.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,537 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,539 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf1/ca7baad3fad14bd491feda5bb5ccd9ba.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,539 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0425965d6b3543f585f733e0e233de1b 2023-11-19 23:53:34,541 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,541 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/0dc0cd2cf9814698b9ba6a4496141d16 2023-11-19 23:53:34,543 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,543 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/0425965d6b3543f585f733e0e233de1b.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,543 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/46a48d4d6953468dae04e436de09137a 2023-11-19 23:53:34,544 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,544 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/0dc0cd2cf9814698b9ba6a4496141d16.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,546 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/7252065028b64ab2862e44e1e62ae969 2023-11-19 23:53:34,546 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,546 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/46a48d4d6953468dae04e436de09137a.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,548 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/74a78c611df34bffa7a4ee5ef7baa220 2023-11-19 23:53:34,548 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,548 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/7252065028b64ab2862e44e1e62ae969.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,549 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/ed53b1ba7b2e43c99920a2ae51a8f510 2023-11-19 23:53:34,550 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/74a78c611df34bffa7a4ee5ef7baa220.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,550 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,552 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/cf2/f428257df22a49a093cee1d6e796f2ac 2023-11-19 23:53:34,552 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/ed53b1ba7b2e43c99920a2ae51a8f510.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,552 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/0c4a4342de5afa19f4fa73194454b024/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,554 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/0b50a5610148ac95f35e1e3c642779e7/cf2/f428257df22a49a093cee1d6e796f2ac.515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,556 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9832be1b663d4774ac3a2edf5626ec08 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf1/9832be1b663d4774ac3a2edf5626ec08 2023-11-19 23:53:34,557 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d/recovered.edits/85.seqid 2023-11-19 23:53:34,557 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/515c2e69ac3f877374dbf5e5cd3d1f8d 2023-11-19 23:53:34,558 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/320b38d58d194c54b2b9b96d9fbbef83 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf1/320b38d58d194c54b2b9b96d9fbbef83 2023-11-19 23:53:34,559 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f37f2291490541778b33bd1a08d62fd1 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/cf2/f37f2291490541778b33bd1a08d62fd1 2023-11-19 23:53:34,561 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/59abfe8fdba5424c966fd94a3f60802d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/cf2/59abfe8fdba5424c966fd94a3f60802d 2023-11-19 23:53:34,563 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024/recovered.edits/90.seqid 2023-11-19 23:53:34,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0c4a4342de5afa19f4fa73194454b024 2023-11-19 23:53:34,564 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7/recovered.edits/90.seqid 2023-11-19 23:53:34,565 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitWithoutPONR/0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:34,565 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-11-19 23:53:34,567 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,570 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-11-19 23:53:34,572 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-11-19 23:53:34,573 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,573 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-11-19 23:53:34,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438014573"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:34,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438014573"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:34,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438014573"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:34,575 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:53:34,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 515c2e69ac3f877374dbf5e5cd3d1f8d, NAME => 'testSplitWithoutPONR,,1700438008120.515c2e69ac3f877374dbf5e5cd3d1f8d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0c4a4342de5afa19f4fa73194454b024, NAME => 'testSplitWithoutPONR,,1700438010130.0c4a4342de5afa19f4fa73194454b024.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0b50a5610148ac95f35e1e3c642779e7, NAME => 'testSplitWithoutPONR,41,1700438010130.0b50a5610148ac95f35e1e3c642779e7.', STARTKEY => '41', ENDKEY => ''}] 2023-11-19 23:53:34,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-11-19 23:53:34,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438014575"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:34,576 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-11-19 23:53:34,580 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:34,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 75 msec 2023-11-19 23:53:34,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-11-19 23:53:34,610 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-11-19 23:53:34,626 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=402 (was 391) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:45044 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:46444 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:45034 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:38298 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:46438 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:33146 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:38342 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:44884 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:32906 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:36283-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:45052 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:45024 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:38404 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4b9cda40-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:33184 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:45098 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:33392 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1251551836_17 at /127.0.0.1:38368 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=888 (was 877) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=328 (was 331), ProcessCount=171 (was 172), AvailableMemoryMB=15282 (was 15293) 2023-11-19 23:53:34,642 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402, OpenFileDescriptor=888, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=171, AvailableMemoryMB=15281 2023-11-19 23:53:34,642 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:34,644 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:34,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:34,646 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:34,648 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:34,649 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:34,650 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:34,651 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 empty. 2023-11-19 23:53:34,651 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:34,651 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-11-19 23:53:34,662 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:34,663 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => ad9b2a7a0288c9e75c03b4a758726fb9, NAME => 'testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:34,672 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:34,672 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing ad9b2a7a0288c9e75c03b4a758726fb9, disabling compactions & flushes 2023-11-19 23:53:34,672 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:34,672 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:34,672 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. after waiting 0 ms 2023-11-19 23:53:34,672 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:34,673 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:34,673 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:34,675 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:34,675 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438014675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438014675"}]},"ts":"1700438014675"} 2023-11-19 23:53:34,677 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:34,678 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:34,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438014678"}]},"ts":"1700438014678"} 2023-11-19 23:53:34,679 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-11-19 23:53:34,682 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:34,683 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:34,683 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:34,683 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:34,683 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:34,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN}] 2023-11-19 23:53:34,684 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:34,685 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:53:34,835 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:34,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:34,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438014836"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438014836"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438014836"}]},"ts":"1700438014836"} 2023-11-19 23:53:34,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:34,991 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:34,991 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-19 23:53:34,993 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56748, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-19 23:53:34,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:34,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ad9b2a7a0288c9e75c03b4a758726fb9, NAME => 'testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:34,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:35,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:35,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,001 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,002 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1 2023-11-19 23:53:35,002 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1 2023-11-19 23:53:35,003 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad9b2a7a0288c9e75c03b4a758726fb9 columnFamilyName cf1 2023-11-19 23:53:35,003 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(310): Store=ad9b2a7a0288c9e75c03b4a758726fb9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:35,003 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,005 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2 2023-11-19 23:53:35,005 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2 2023-11-19 23:53:35,005 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad9b2a7a0288c9e75c03b4a758726fb9 columnFamilyName cf2 2023-11-19 23:53:35,006 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(310): Store=ad9b2a7a0288c9e75c03b4a758726fb9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:35,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:35,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:35,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:35,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ad9b2a7a0288c9e75c03b4a758726fb9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67958426, jitterRate=0.012659460306167603}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:35,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:35,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9., pid=43, masterSystemTime=1700438014990 2023-11-19 23:53:35,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438015018"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438015018"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438015018"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438015018"}]},"ts":"1700438015018"} 2023-11-19 23:53:35,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=43, resume processing ppid=42 2023-11-19 23:53:35,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 in 181 msec 2023-11-19 23:53:35,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=42, resume processing ppid=41 2023-11-19 23:53:35,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN in 338 msec 2023-11-19 23:53:35,023 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:35,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438015024"}]},"ts":"1700438015024"} 2023-11-19 23:53:35,025 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-11-19 23:53:35,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:35,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 383 msec 2023-11-19 23:53:35,149 DEBUG [Listener at localhost/42385] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-19 23:53:35,155 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-19 23:53:35,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:35,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:35,171 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,171 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,171 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:35,171 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:35,172 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,172 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,173 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,174 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,174 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,174 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:35,177 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,177 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:35,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,181 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,183 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:53:35,183 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,183 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,185 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,187 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,187 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,187 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,244 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/cad42526483f4cf08729a77459b5c1ae 2023-11-19 23:53:35,276 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/51037d467f804a60ac3ff707d76e3c21 2023-11-19 23:53:35,281 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/cad42526483f4cf08729a77459b5c1ae as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae 2023-11-19 23:53:35,287 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:35,288 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/51037d467f804a60ac3ff707d76e3c21 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 2023-11-19 23:53:35,293 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:35,294 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 111ms, sequenceid=6, compaction requested=false 2023-11-19 23:53:35,294 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,295 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:35,295 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:35,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:35,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,295 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,297 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,297 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,297 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,298 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:35,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@98ff234[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:35,301 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:35,301 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,302 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:35,303 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,303 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:35,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,303 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:35,305 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,305 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,305 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,306 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,306 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,306 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,307 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,306 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:53:35,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:35,312 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:35,312 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:35,312 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:35,312 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:35,312 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:35,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,317 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,317 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,318 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,318 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,320 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,321 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,321 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,324 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,324 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:35,325 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:35,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,426 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:35,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:35,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:35,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:35,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:35,449 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:35,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:35,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,450 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,451 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,452 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,452 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:35,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,453 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,453 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,453 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:35,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:35,455 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,455 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:35,455 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:35,456 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,457 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,471 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d 2023-11-19 23:53:35,495 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/6325acd2a5524c5890bc405deb0fd60e 2023-11-19 23:53:35,501 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d 2023-11-19 23:53:35,506 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:35,507 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/6325acd2a5524c5890bc405deb0fd60e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e 2023-11-19 23:53:35,511 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:35,512 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 57ms, sequenceid=19, compaction requested=false 2023-11-19 23:53:35,512 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:35,512 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,512 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:35,512 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:35,512 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,512 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,512 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,515 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:35,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c4b7185[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:35,518 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:35,518 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,519 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,519 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:35,520 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,520 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,520 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,520 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:35,520 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,520 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:35,520 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:35,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:53:35,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:35,526 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:35,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:35,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:35,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:35,526 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:35,529 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,529 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,529 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,530 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,530 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,530 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,531 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,532 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,534 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,534 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,534 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,539 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:35,538 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,539 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,639 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:35,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:35,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:35,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:35,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:35,656 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,656 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,656 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:35,656 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:35,656 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,656 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,660 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,660 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,660 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,660 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,661 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,661 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,662 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,662 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,663 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,663 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:35,664 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,677 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/eee62c70cfe6488eaa7940471d801939 2023-11-19 23:53:35,691 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/34c6005edbf345ce9f91db62948d0de1 2023-11-19 23:53:35,696 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/eee62c70cfe6488eaa7940471d801939 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 2023-11-19 23:53:35,700 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:35,701 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/34c6005edbf345ce9f91db62948d0de1 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 2023-11-19 23:53:35,705 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:35,706 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 43ms, sequenceid=32, compaction requested=true 2023-11-19 23:53:35,706 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:35,706 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,709 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,709 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,709 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,709 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:35,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c4d65a[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:35,712 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:35,712 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,713 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,714 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:35,715 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,715 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,715 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,715 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:35,715 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,715 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,715 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,717 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,717 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:35,717 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:35,717 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,717 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:35,717 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,718 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:35,718 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:35,718 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:35,718 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:35,719 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,719 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,719 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,719 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,719 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,720 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,721 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,721 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,722 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,724 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,725 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,725 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,729 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,728 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:35,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,729 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,730 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,830 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:35,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:35,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:35,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:35,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:35,855 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:35,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:35,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,856 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,858 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,858 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:35,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,862 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,862 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,862 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:35,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:35,863 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,864 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:35,864 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:35,865 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,865 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,865 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,879 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/e3015f02b3e54270b0793eb52d194cf9 2023-11-19 23:53:35,890 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/93613a3e73de4643a422196844e2c1e9 2023-11-19 23:53:35,894 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/e3015f02b3e54270b0793eb52d194cf9 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 2023-11-19 23:53:35,899 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:35,900 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/93613a3e73de4643a422196844e2c1e9 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 2023-11-19 23:53:35,903 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:35,904 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 40ms, sequenceid=45, compaction requested=true 2023-11-19 23:53:35,904 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:35,904 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:35,906 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:35,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73844990[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:35,910 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:35,910 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,912 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,912 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,912 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:35,912 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,912 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:35,913 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,921 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:35,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:53:35,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:35,925 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:35,925 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:35,925 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:35,925 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:35,925 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:35,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,928 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,928 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:35,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:35,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:35,931 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:35,931 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,932 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:35,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:35,935 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:35,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:35,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:35,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:35,936 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:35,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:35,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,036 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:36,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:36,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:36,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:36,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:36,055 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:36,055 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,055 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:36,055 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:36,055 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,055 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,057 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,058 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,058 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:36,060 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,060 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,060 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,060 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:36,063 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,063 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,063 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,064 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,064 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,064 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,071 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/d2180806e03d485e86316c61cd3a1b0b 2023-11-19 23:53:36,084 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/d64e9732ba6e4347bd254c5823ee7b48 2023-11-19 23:53:36,088 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/d2180806e03d485e86316c61cd3a1b0b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b 2023-11-19 23:53:36,092 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:36,093 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/d64e9732ba6e4347bd254c5823ee7b48 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 2023-11-19 23:53:36,098 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:36,099 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 39ms, sequenceid=58, compaction requested=true 2023-11-19 23:53:36,099 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:36,099 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:36,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56c1b86[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:36,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:36,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,105 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,105 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:36,105 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,108 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,108 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:53:36,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:36,112 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:36,112 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:36,112 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:36,112 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:36,112 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:36,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,114 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,114 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,115 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,115 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,117 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,117 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,118 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:36,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,124 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:36,124 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,125 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,225 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:36,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:36,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:36,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:36,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-11-19 23:53:36,241 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-11-19 23:53:36,241 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,241 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-11-19 23:53:36,241 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:53:36,242 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,242 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,284 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,285 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-11-19 23:53:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:36,287 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:36,287 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:36,287 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. started... 2023-11-19 23:53:36,287 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:36,296 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/6c88e3d079e448e5b390c119d94dc8b6 2023-11-19 23:53:36,308 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 2023-11-19 23:53:36,313 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/6c88e3d079e448e5b390c119d94dc8b6 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 2023-11-19 23:53:36,317 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:36,318 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 2023-11-19 23:53:36,322 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:36,323 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 35ms, sequenceid=71, compaction requested=true 2023-11-19 23:53:36,323 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:36,323 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:36,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:36,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:36,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-11-19 23:53:36,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-11-19 23:53:36,330 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,430 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-11-19 23:53:36,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72dc85f7[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:36,433 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:53:36,433 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,477 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,477 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:36,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:36,478 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:53:36,478 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-11-19 23:53:36,478 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:36,523 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,523 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,523 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,523 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-11-19 23:53:36,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,546 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-11-19 23:53:36,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:53:36,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:53:36,550 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-11-19 23:53:36,550 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:36,550 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:36,550 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-11-19 23:53:36,550 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:36,584 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,585 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:36,585 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,585 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,586 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:36,588 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-11-19 23:53:36,588 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,588 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,661 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,661 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,661 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,662 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:36,661 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:36,661 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:36,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-11-19 23:53:36,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,662 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:36,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,663 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:36,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:36,663 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:36,663 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-11-19 23:53:36,663 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-11-19 23:53:36,763 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:36,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:37,023 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-11-19 23:53:37,024 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:37,025 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:37,027 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:37,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:37,275 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,275 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:37,275 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:37,275 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:37,275 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-11-19 23:53:37,275 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:37,276 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:37,276 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:37,276 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:37,276 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:37,277 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-11-19 23:53:37,278 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:37,278 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:37,278 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:37,278 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:37,279 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,279 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:37,279 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,279 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:37,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:37,281 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,281 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:37,281 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,281 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,281 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,281 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:37,282 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:37,282 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:37,283 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438047283 2023-11-19 23:53:37,284 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,284 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:37,286 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:37,287 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:37,287 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:37,287 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438077287 2023-11-19 23:53:37,287 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438137287 2023-11-19 23:53:37,287 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:53:37,287 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:37,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:37,289 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:37,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:37,537 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,537 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:37,537 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:37,537 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:37,539 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:37,539 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:37,539 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:37,539 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:37,539 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:37,540 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:37,540 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-11-19 23:53:37,541 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:37,541 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:37,541 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:37,542 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:37,542 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,542 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:37,542 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,543 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,543 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,543 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:37,543 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,543 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,544 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:37,544 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,544 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:37,544 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,544 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,544 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,545 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:37,545 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:37,546 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:37,546 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438047546 2023-11-19 23:53:37,548 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,548 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:37,550 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:37,550 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:37,550 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:37,550 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438077550 2023-11-19 23:53:37,550 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438137550 2023-11-19 23:53:37,550 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:53:37,551 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:37,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN}] 2023-11-19 23:53:37,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:37,552 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:37,553 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:37,801 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,801 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:37,801 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:37,801 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:37,803 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:37,803 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:37,803 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:37,803 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:37,803 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:37,804 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:37,805 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-11-19 23:53:37,805 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:37,806 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:37,806 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:37,806 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:37,806 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,807 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:37,807 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,807 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:37,807 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,807 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:37,808 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,808 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:37,809 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:37,809 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,809 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:37,809 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,809 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,809 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,809 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:37,810 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:37,810 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:37,811 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438047811 2023-11-19 23:53:37,811 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:37,812 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:37,814 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:37,814 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:37,814 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:37,814 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438077814 2023-11-19 23:53:37,815 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438137815 2023-11-19 23:53:37,815 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:37,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN}] 2023-11-19 23:53:37,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:37,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:37,818 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:37,818 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438017818"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438017818"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438017818"}]},"ts":"1700438017818"} 2023-11-19 23:53:37,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:37,819 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:37,819 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:37,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:37,820 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:38,065 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:38,065 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:38,066 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:38,066 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:38,066 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:38,067 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:38,067 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:38,068 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:38,068 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:38,068 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:38,068 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:38,069 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,069 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:38,069 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,069 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,070 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,070 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:38,070 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,070 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,070 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:38,071 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,071 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,071 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,071 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,071 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,071 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,072 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,072 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,072 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:38,072 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:38,073 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:38,073 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:38,073 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438048073 2023-11-19 23:53:38,074 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,074 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:38,076 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:38,076 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:38,076 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:38,076 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438078076 2023-11-19 23:53:38,076 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438138076 2023-11-19 23:53:38,077 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:53:38,077 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,077 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438018077"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438018077"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438018077"}]},"ts":"1700438018077"} 2023-11-19 23:53:38,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:38,079 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:38,231 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:38,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ad9b2a7a0288c9e75c03b4a758726fb9, disabling compactions & flushes 2023-11-19 23:53:38,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:38,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:38,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. after waiting 0 ms 2023-11-19 23:53:38,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:38,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ad9b2a7a0288c9e75c03b4a758726fb9 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:53:38,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/464a0ec9e3154e808e1e607ad4080953 2023-11-19 23:53:38,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/546e3e3ee62a45caa5f07f51baf632e8 2023-11-19 23:53:38,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/464a0ec9e3154e808e1e607ad4080953 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 2023-11-19 23:53:38,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:38,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/546e3e3ee62a45caa5f07f51baf632e8 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 2023-11-19 23:53:38,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:53:38,271 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ad9b2a7a0288c9e75c03b4a758726fb9 in 39ms, sequenceid=82, compaction requested=true 2023-11-19 23:53:38,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:53:38,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:38,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:38,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:38,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:38,279 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED 2023-11-19 23:53:38,279 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438018279"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438018279"}]},"ts":"1700438018279"} 2023-11-19 23:53:38,328 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:38,328 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:38,328 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:38,577 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,577 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:38,577 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:38,577 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:38,577 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:38,577 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:38,578 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:38,578 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:38,578 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:38,578 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:38,579 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:38,580 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-11-19 23:53:38,580 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-11-19 23:53:38,580 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:38,580 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:38,580 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:38,581 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:38,581 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,581 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:38,581 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,581 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,582 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,582 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:38,582 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,582 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,583 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:38,583 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,583 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,583 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,583 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,583 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,583 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,584 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,584 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,584 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,584 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,584 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:38,585 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:38,586 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:38,586 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:38,586 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438048586 2023-11-19 23:53:38,586 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:38,587 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,590 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:38,590 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:38,590 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:38,590 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438078590 2023-11-19 23:53:38,590 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438138590 2023-11-19 23:53:38,590 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:38,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438018590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438018590"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438018590"}]},"ts":"1700438018590"} 2023-11-19 23:53:38,592 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:38,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=46, resume processing ppid=45 2023-11-19 23:53:38,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 in 514 msec 2023-11-19 23:53:38,595 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:38,595 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:38,595 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:38,840 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,841 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:38,841 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:38,841 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:38,842 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:38,842 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:38,842 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:38,842 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:38,842 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:38,843 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:38,843 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:38,844 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-11-19 23:53:38,844 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:38,844 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:38,845 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:38,845 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:38,845 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:38,845 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,846 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:38,846 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,846 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:38,846 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,846 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:38,847 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,847 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:38,847 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:38,848 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,848 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,848 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:38,848 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:38,849 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:38,849 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:38,850 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:38,850 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:38,850 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438048850 2023-11-19 23:53:38,850 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:38,851 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,853 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:38,853 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:38,853 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:38,853 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438078853 2023-11-19 23:53:38,853 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438138853 2023-11-19 23:53:38,853 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:38,854 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:38,855 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:38,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=45, resume processing ppid=44 2023-11-19 23:53:38,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN in 1.0400 sec 2023-11-19 23:53:38,860 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:38,860 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:38,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:38,860 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:39,104 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,104 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:39,104 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:39,104 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:39,105 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:39,105 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:39,105 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:39,105 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:39,105 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:39,106 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:39,107 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:39,107 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:39,107 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:39,107 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:39,108 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:39,108 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:39,108 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:39,109 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,109 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:39,109 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,109 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,110 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,110 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:39,110 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,111 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,111 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:39,112 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:39,113 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:39,113 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:39,114 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:39,118 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438049118 2023-11-19 23:53:39,119 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:39,119 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,122 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:39,122 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:39,122 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:39,122 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438079122 2023-11-19 23:53:39,122 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438139122 2023-11-19 23:53:39,122 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:39,122 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,122 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,122 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:39,129 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:53:39,138 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ad9b2a7a0288c9e75c03b4a758726fb9, threads=14 2023-11-19 23:53:39,144 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,145 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,145 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,145 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,145 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,147 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,148 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,152 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,152 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,153 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,153 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,160 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,162 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,162 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,348 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,356 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,374 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,374 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,401 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,401 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,411 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,416 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,416 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,418 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,418 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,419 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,420 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,423 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,423 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ad9b2a7a0288c9e75c03b4a758726fb9 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:39,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:39,461 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:39,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:39,622 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:39,623 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:39,623 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:39,624 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:39,625 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-11-19 23:53:39,625 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:39,625 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:39,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:39,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:39,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:39,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:39,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:39,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:39,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:39,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:39,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:39,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:39,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:39,629 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:39,630 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:39,631 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:39,631 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438049631 2023-11-19 23:53:39,632 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:39,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:39,635 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:39,635 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:39,635 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:39,635 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438079635 2023-11-19 23:53:39,635 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438139635 2023-11-19 23:53:39,635 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:39,647 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ad9b2a7a0288c9e75c03b4a758726fb9, threads=14 2023-11-19 23:53:39,648 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,648 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,648 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,648 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,649 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,649 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,649 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,650 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,651 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,835 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,856 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,880 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,885 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,885 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,885 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,885 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,888 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,892 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,893 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,900 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:39,902 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:40,247 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:40,251 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 for region: ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:40,251 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ad9b2a7a0288c9e75c03b4a758726fb9 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:40,285 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:40,290 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/4c377b2f8ce4a84a38366317178f7b91/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:40,292 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/380f57fc7e4f191d259216af1345c0d9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:53:40,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:40,292 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:40,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:40,386 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,386 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:40,386 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:40,386 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:40,386 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:40,387 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:40,387 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:40,387 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:40,387 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:40,387 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:40,388 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-11-19 23:53:40,388 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:40,388 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:40,388 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:40,389 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:40,389 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:40,389 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:40,389 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:40,390 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:40,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:40,391 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:40,393 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:40,393 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,393 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,393 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,393 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,393 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,393 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:40,394 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:40,394 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:40,395 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438050395 2023-11-19 23:53:40,395 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:40,396 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,398 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:40,398 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:40,398 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:40,398 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438080398 2023-11-19 23:53:40,398 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438140398 2023-11-19 23:53:40,398 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:53:40,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:40,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:40,403 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:40,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:40,649 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,649 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:40,649 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:40,649 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:40,650 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:40,650 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:40,650 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:40,650 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:40,650 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:40,651 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:40,652 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-11-19 23:53:40,652 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:40,652 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:40,653 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:40,653 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:40,653 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:40,654 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:40,654 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,654 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:40,654 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,654 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,655 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,655 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:40,655 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,655 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,656 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:40,657 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,657 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,657 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,657 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,658 DEBUG [Listener at localhost/42385] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,658 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:53:40,658 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:40,659 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:40,659 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438050659 2023-11-19 23:53:40,659 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:40,662 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,664 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:40,664 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:40,664 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:40,664 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438080664 2023-11-19 23:53:40,664 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438140664 2023-11-19 23:53:40,664 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 5 msec 2023-11-19 23:53:40,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:40,677 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:40,677 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:40,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:40,677 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:40,916 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,916 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:40,916 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:40,916 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:40,917 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:40,917 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:40,917 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:40,917 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:40,917 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:40,918 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:40,918 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:40,919 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:40,920 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,920 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:40,920 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,920 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:40,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:40,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,921 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:40,922 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:40,922 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,923 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:40,923 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:40,923 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:40,923 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,923 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,923 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,923 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,923 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:40,923 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:40,924 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:40,924 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:40,925 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438050925 2023-11-19 23:53:40,927 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:40,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:40,930 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:40,930 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:40,930 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:40,930 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438080930 2023-11-19 23:53:40,930 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438140930 2023-11-19 23:53:40,930 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:40,930 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:40,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:40,932 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:40,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:40,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:41,181 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,181 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:41,181 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:41,181 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:41,182 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:41,182 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:41,182 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:41,182 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:41,182 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:41,183 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:41,184 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-11-19 23:53:41,184 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:41,184 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:41,184 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:41,185 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:41,185 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:41,185 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:41,185 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,185 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,186 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:41,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,187 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:41,188 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:41,188 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:41,188 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,188 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,188 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,188 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,188 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,188 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:41,189 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:41,189 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:41,190 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438051190 2023-11-19 23:53:41,190 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:41,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,193 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:41,193 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:41,193 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:41,193 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438081193 2023-11-19 23:53:41,193 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438141193 2023-11-19 23:53:41,194 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:41,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:41,196 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:41,196 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:41,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:41,196 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:41,444 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:41,444 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:41,445 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:41,446 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-11-19 23:53:41,446 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:41,446 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:41,446 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:41,447 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:41,447 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:41,447 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:41,447 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:41,448 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,449 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,449 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:41,449 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,449 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:41,449 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:41,450 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:41,450 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,450 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,450 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,450 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,450 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,450 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:41,451 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:41,451 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:41,451 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438051451 2023-11-19 23:53:41,451 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:41,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,455 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:41,455 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:41,455 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:41,455 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438081455 2023-11-19 23:53:41,455 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438141455 2023-11-19 23:53:41,455 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:41,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:41,459 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1053): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN] 2023-11-19 23:53:41,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:41,459 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:41,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:41,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:41,706 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,706 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:41,706 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:41,706 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:41,708 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:41,708 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:41,708 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:41,709 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:41,709 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:41,709 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:41,710 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-11-19 23:53:41,710 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:53:41,710 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:53:41,710 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:41,710 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:41,711 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:41,711 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:41,711 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:41,711 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:41,712 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,713 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,713 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:41,714 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,714 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,714 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,714 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,715 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,715 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:41,715 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:41,716 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:41,716 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:41,716 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438051716 2023-11-19 23:53:41,717 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:41,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,718 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:41,720 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:41,720 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:41,721 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:41,721 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438081721 2023-11-19 23:53:41,721 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438141721 2023-11-19 23:53:41,721 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:41,721 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:41,725 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:41,725 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:41,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:41,725 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:41,971 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,971 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:41,971 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:41,971 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:41,971 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:41,972 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:41,972 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:41,972 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:41,972 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:41,972 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:41,973 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:53:41,973 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:41,973 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:41,974 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:41,974 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:41,974 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:41,974 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,974 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,975 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:41,976 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:41,976 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,976 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:41,976 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:41,976 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,976 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,976 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,976 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:41,976 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 3 msec 2023-11-19 23:53:41,977 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:41,978 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:41,978 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:41,978 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438051978 2023-11-19 23:53:41,979 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:41,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:41,982 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:41,983 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:41,983 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:41,983 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:41,983 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438081983 2023-11-19 23:53:41,984 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438141984 2023-11-19 23:53:41,984 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:53:41,984 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:41,986 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:41,986 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:41,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:41,986 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:42,235 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:42,236 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:42,236 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:42,237 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:42,238 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-11-19 23:53:42,238 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:42,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:42,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,239 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,240 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,240 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:42,240 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,240 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,240 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:42,241 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,241 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,241 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:42,241 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,241 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,241 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,241 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:42,241 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 4 msec 2023-11-19 23:53:42,242 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:42,242 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:42,243 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:42,243 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438052243 2023-11-19 23:53:42,243 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:42,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,246 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,248 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:42,248 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:42,248 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:42,248 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438082248 2023-11-19 23:53:42,248 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438142248 2023-11-19 23:53:42,248 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 5 msec 2023-11-19 23:53:42,249 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:42,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:42,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1580): TESTING: Kill before store update 2023-11-19 23:53:42,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:42,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1582) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1497) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1348) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:42,499 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,499 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:42,499 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:42,499 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:42,499 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:42,499 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:42,500 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:42,500 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:42,500 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:42,500 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:42,501 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-11-19 23:53:42,501 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-11-19 23:53:42,502 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:42,502 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:42,502 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:42,502 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:42,503 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,503 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,504 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,505 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:42,505 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,505 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,505 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:42,505 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,506 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,506 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,506 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:53:42,506 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:42,506 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:42,507 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:42,507 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:42,508 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438052508 2023-11-19 23:53:42,509 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:42,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1356): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,512 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,514 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:42,514 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:42,515 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:42,515 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438082515 2023-11-19 23:53:42,515 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438142515 2023-11-19 23:53:42,515 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 5 msec 2023-11-19 23:53:42,515 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:53:42,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1527): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 exec-time=5.4910 sec 2023-11-19 23:53:42,518 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,519 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=true 2023-11-19 23:53:42,669 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:42,670 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:42,670 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438022670"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438022670"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438022670"}]},"ts":"1700438022670"} 2023-11-19 23:53:42,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:42,672 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:53:42,672 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:53:42,672 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:53:42,765 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:42,765 WARN [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-11-19 23:53:42,765 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:53:42,765 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:53:42,765 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:53:42,765 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:53:42,766 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:53:42,766 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:53:42,766 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:53:42,766 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:53:42,766 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:53:42,767 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-11-19 23:53:42,768 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:53:42,768 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:53:42,768 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:53:42,768 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:53:42,768 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,769 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:53:42,769 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,769 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:53:42,769 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,770 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:53:42,770 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,770 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:53:42,770 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:42,771 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:53:42,771 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,771 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:53:42,771 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,772 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 5 msec 2023-11-19 23:53:42,772 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:53:42,773 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:53:42,774 INFO [Listener at localhost/42385] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9 to restore RIT 2023-11-19 23:53:42,774 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438052774 2023-11-19 23:53:42,774 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:53:42,775 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:53:42,777 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:53:42,777 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,34057,1700438000374, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374, openSeqNum=2 2023-11-19 23:53:42,778 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:53:42,778 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438082778 2023-11-19 23:53:42,778 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438142778 2023-11-19 23:53:42,778 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:53:42,778 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:42,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438022778"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438022778"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438022778"}]},"ts":"1700438022778"} 2023-11-19 23:53:42,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:42,932 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:42,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:42,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ad9b2a7a0288c9e75c03b4a758726fb9, NAME => 'testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:42,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:42,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:42,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:42,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:42,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:42,937 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:42,938 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1 2023-11-19 23:53:42,938 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1 2023-11-19 23:53:42,939 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad9b2a7a0288c9e75c03b4a758726fb9 columnFamilyName cf1 2023-11-19 23:53:42,946 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 2023-11-19 23:53:42,949 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 2023-11-19 23:53:42,952 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae 2023-11-19 23:53:42,956 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b 2023-11-19 23:53:42,959 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d 2023-11-19 23:53:42,965 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 2023-11-19 23:53:42,968 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 2023-11-19 23:53:42,968 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(310): Store=ad9b2a7a0288c9e75c03b4a758726fb9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:42,969 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:42,970 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2 2023-11-19 23:53:42,970 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2 2023-11-19 23:53:42,970 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ad9b2a7a0288c9e75c03b4a758726fb9 columnFamilyName cf2 2023-11-19 23:53:42,984 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 2023-11-19 23:53:42,989 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 2023-11-19 23:53:42,992 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 2023-11-19 23:53:42,995 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e 2023-11-19 23:53:42,998 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 2023-11-19 23:53:43,001 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 2023-11-19 23:53:43,005 DEBUG [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 2023-11-19 23:53:43,005 INFO [StoreOpener-ad9b2a7a0288c9e75c03b4a758726fb9-1] regionserver.HStore(310): Store=ad9b2a7a0288c9e75c03b4a758726fb9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:43,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.splits 2023-11-19 23:53:43,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:43,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ad9b2a7a0288c9e75c03b4a758726fb9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70720562, jitterRate=0.05381849408149719}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:43,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:43,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9., pid=48, masterSystemTime=1700438022932 2023-11-19 23:53:43,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:43,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:43,013 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:43,013 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:43,014 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-11-19 23:53:43,014 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-11-19 23:53:43,014 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1912): ad9b2a7a0288c9e75c03b4a758726fb9/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:43,014 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1912): ad9b2a7a0288c9e75c03b4a758726fb9/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:43,015 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ad9b2a7a0288c9e75c03b4a758726fb9/cf1 in testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,015 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ad9b2a7a0288c9e75c03b4a758726fb9/cf2 in testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,015 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp, totalSize=34.9 K 2023-11-19 23:53:43,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,015 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp, totalSize=34.9 K 2023-11-19 23:53:43,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,015 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting 51037d467f804a60ac3ff707d76e3c21, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438015155 2023-11-19 23:53:43,015 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting cad42526483f4cf08729a77459b5c1ae, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438015155 2023-11-19 23:53:43,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,016 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438023015"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438023015"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438023015"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438023015"}]},"ts":"1700438023015"} 2023-11-19 23:53:43,016 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting 6325acd2a5524c5890bc405deb0fd60e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438015428 2023-11-19 23:53:43,016 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting e015f8efe9cd49d5b4d7a6105c5a7b2d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438015428 2023-11-19 23:53:43,016 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting 34c6005edbf345ce9f91db62948d0de1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438015640 2023-11-19 23:53:43,016 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting eee62c70cfe6488eaa7940471d801939, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438015640 2023-11-19 23:53:43,017 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting 93613a3e73de4643a422196844e2c1e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438015832 2023-11-19 23:53:43,017 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting e3015f02b3e54270b0793eb52d194cf9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438015832 2023-11-19 23:53:43,017 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting d64e9732ba6e4347bd254c5823ee7b48, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438016038 2023-11-19 23:53:43,017 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting d2180806e03d485e86316c61cd3a1b0b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438016038 2023-11-19 23:53:43,018 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting e40c7e9ecfbf4465ab04ef3c70c0f006, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438016226 2023-11-19 23:53:43,018 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting 6c88e3d079e448e5b390c119d94dc8b6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438016226 2023-11-19 23:53:43,018 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting 546e3e3ee62a45caa5f07f51baf632e8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438016764 2023-11-19 23:53:43,018 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting 464a0ec9e3154e808e1e607ad4080953, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438016764 2023-11-19 23:53:43,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=48, resume processing ppid=47 2023-11-19 23:53:43,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 in 237 msec 2023-11-19 23:53:43,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN in 1.5620 sec 2023-11-19 23:53:43,029 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-11-19 23:53:43,059 DEBUG [Listener at localhost/42385] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-11-19 23:53:43,061 INFO [Listener at localhost/42385] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-11-19 23:53:43,069 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-11-19 23:53:43,069 INFO [Listener at localhost/42385] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-11-19 23:53:43,070 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-11-19 23:53:43,072 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] throttle.PressureAwareThroughputController(145): ad9b2a7a0288c9e75c03b4a758726fb9#cf2#compaction#50 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:43,081 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] throttle.PressureAwareThroughputController(145): ad9b2a7a0288c9e75c03b4a758726fb9#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:43,094 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf2/514dbc9eb33f462b8d24d8e9ff7f6c83 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/514dbc9eb33f462b8d24d8e9ff7f6c83 2023-11-19 23:53:43,098 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/.tmp/cf1/2ace2c2d43d14eb0a060db6ccbbc99ce as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/2ace2c2d43d14eb0a060db6ccbbc99ce 2023-11-19 23:53:43,102 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ad9b2a7a0288c9e75c03b4a758726fb9/cf2 of ad9b2a7a0288c9e75c03b4a758726fb9 into 514dbc9eb33f462b8d24d8e9ff7f6c83(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:43,102 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:43,102 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9., storeName=ad9b2a7a0288c9e75c03b4a758726fb9/cf2, priority=9, startTime=1700438023012; duration=0sec 2023-11-19 23:53:43,103 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:43,108 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ad9b2a7a0288c9e75c03b4a758726fb9/cf1 of ad9b2a7a0288c9e75c03b4a758726fb9 into 2ace2c2d43d14eb0a060db6ccbbc99ce(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:43,108 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:43,108 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9., storeName=ad9b2a7a0288c9e75c03b4a758726fb9/cf1, priority=9, startTime=1700438023011; duration=0sec 2023-11-19 23:53:43,108 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:43,117 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:43,119 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-11-19 23:53:43,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-11-19 23:53:43,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-19 23:53:43,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438023124"}]},"ts":"1700438023124"} 2023-11-19 23:53:43,126 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-11-19 23:53:43,128 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-11-19 23:53:43,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN}] 2023-11-19 23:53:43,130 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN 2023-11-19 23:53:43,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438023130"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438023130"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438023130"}]},"ts":"1700438023130"} 2023-11-19 23:53:43,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:43,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-19 23:53:43,283 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ad9b2a7a0288c9e75c03b4a758726fb9, disabling compactions & flushes 2023-11-19 23:53:43,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. after waiting 0 ms 2023-11-19 23:53:43,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,285 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953] to archive 2023-11-19 23:53:43,286 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:43,288 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/cad42526483f4cf08729a77459b5c1ae 2023-11-19 23:53:43,289 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e015f8efe9cd49d5b4d7a6105c5a7b2d 2023-11-19 23:53:43,290 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/eee62c70cfe6488eaa7940471d801939 2023-11-19 23:53:43,291 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/e3015f02b3e54270b0793eb52d194cf9 2023-11-19 23:53:43,292 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/d2180806e03d485e86316c61cd3a1b0b 2023-11-19 23:53:43,294 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/6c88e3d079e448e5b390c119d94dc8b6 2023-11-19 23:53:43,295 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/464a0ec9e3154e808e1e607ad4080953 2023-11-19 23:53:43,299 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8] to archive 2023-11-19 23:53:43,300 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:43,302 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/51037d467f804a60ac3ff707d76e3c21 2023-11-19 23:53:43,304 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/6325acd2a5524c5890bc405deb0fd60e 2023-11-19 23:53:43,305 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/34c6005edbf345ce9f91db62948d0de1 2023-11-19 23:53:43,306 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/93613a3e73de4643a422196844e2c1e9 2023-11-19 23:53:43,307 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/d64e9732ba6e4347bd254c5823ee7b48 2023-11-19 23:53:43,308 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/e40c7e9ecfbf4465ab04ef3c70c0f006 2023-11-19 23:53:43,309 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/546e3e3ee62a45caa5f07f51baf632e8 2023-11-19 23:53:43,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:53:43,314 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:43,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9. 2023-11-19 23:53:43,314 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ad9b2a7a0288c9e75c03b4a758726fb9: 2023-11-19 23:53:43,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ad9b2a7a0288c9e75c03b4a758726fb9, regionState=CLOSED 2023-11-19 23:53:43,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438023316"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438023316"}]},"ts":"1700438023316"} 2023-11-19 23:53:43,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=51, resume processing ppid=50 2023-11-19 23:53:43,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure ad9b2a7a0288c9e75c03b4a758726fb9, server=jenkins-hbase4.apache.org,34057,1700438000374 in 185 msec 2023-11-19 23:53:43,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=50, resume processing ppid=49 2023-11-19 23:53:43,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, UNASSIGN in 190 msec 2023-11-19 23:53:43,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438023321"}]},"ts":"1700438023321"} 2023-11-19 23:53:43,323 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-11-19 23:53:43,325 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-11-19 23:53:43,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 206 msec 2023-11-19 23:53:43,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-19 23:53:43,426 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-11-19 23:53:43,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-11-19 23:53:43,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,431 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-11-19 23:53:43,435 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,438 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits] 2023-11-19 23:53:43,441 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/2ace2c2d43d14eb0a060db6ccbbc99ce to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf1/2ace2c2d43d14eb0a060db6ccbbc99ce 2023-11-19 23:53:43,444 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/514dbc9eb33f462b8d24d8e9ff7f6c83 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/cf2/514dbc9eb33f462b8d24d8e9ff7f6c83 2023-11-19 23:53:43,447 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9/recovered.edits/90.seqid 2023-11-19 23:53:43,448 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackAndDoubleExecution/ad9b2a7a0288c9e75c03b4a758726fb9 2023-11-19 23:53:43,448 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-11-19 23:53:43,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,453 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-11-19 23:53:43,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-11-19 23:53:43,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-11-19 23:53:43,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438023455"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:43,457 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-19 23:53:43,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ad9b2a7a0288c9e75c03b4a758726fb9, NAME => 'testRollbackAndDoubleExecution,,1700438014645.ad9b2a7a0288c9e75c03b4a758726fb9.', STARTKEY => '', ENDKEY => ''}] 2023-11-19 23:53:43,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-11-19 23:53:43,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438023457"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:43,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-11-19 23:53:43,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:53:43,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 34 msec 2023-11-19 23:53:43,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-11-19 23:53:43,533 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-11-19 23:53:43,549 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=405 (was 402) Potentially hanging thread: RS:1;jenkins-hbase4:34057-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48254 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48230 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48232 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:38298 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:38342 [Waiting for operation #57] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:37222 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48200 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:37218 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48224 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48204 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:48210 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:37206 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:38404 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:37192 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:37228 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2082464960_17 at /127.0.0.1:38368 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=886 (was 888), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=365 (was 328) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=15205 (was 15281) 2023-11-19 23:53:43,562 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=405, OpenFileDescriptor=886, MaxFileDescriptor=60000, SystemLoadAverage=365, ProcessCount=171, AvailableMemoryMB=15204 2023-11-19 23:53:43,562 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:43,564 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:43,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:43,566 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:43,567 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:43,568 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:43,570 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,570 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 empty. 2023-11-19 23:53:43,571 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,571 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-11-19 23:53:43,581 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:43,582 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5876f55847b223c4fd0e2c973b172fc6, NAME => 'testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:43,593 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:43,593 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 5876f55847b223c4fd0e2c973b172fc6, disabling compactions & flushes 2023-11-19 23:53:43,593 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,593 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,593 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. after waiting 0 ms 2023-11-19 23:53:43,594 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,594 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,594 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 5876f55847b223c4fd0e2c973b172fc6: 2023-11-19 23:53:43,596 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:43,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438023597"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438023597"}]},"ts":"1700438023597"} 2023-11-19 23:53:43,599 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:43,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:43,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438023600"}]},"ts":"1700438023600"} 2023-11-19 23:53:43,601 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-11-19 23:53:43,605 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:43,606 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:43,606 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:43,606 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:43,606 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:43,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, ASSIGN}] 2023-11-19 23:53:43,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, ASSIGN 2023-11-19 23:53:43,609 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:53:43,759 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:43,760 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=5876f55847b223c4fd0e2c973b172fc6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,760 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438023760"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438023760"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438023760"}]},"ts":"1700438023760"} 2023-11-19 23:53:43,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 5876f55847b223c4fd0e2c973b172fc6, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:43,914 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5876f55847b223c4fd0e2c973b172fc6, NAME => 'testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:43,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:43,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:43,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,921 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,922 DEBUG [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf1 2023-11-19 23:53:43,922 DEBUG [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf1 2023-11-19 23:53:43,924 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5876f55847b223c4fd0e2c973b172fc6 columnFamilyName cf1 2023-11-19 23:53:43,925 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] regionserver.HStore(310): Store=5876f55847b223c4fd0e2c973b172fc6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:43,925 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,926 DEBUG [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf2 2023-11-19 23:53:43,926 DEBUG [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf2 2023-11-19 23:53:43,927 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5876f55847b223c4fd0e2c973b172fc6 columnFamilyName cf2 2023-11-19 23:53:43,927 INFO [StoreOpener-5876f55847b223c4fd0e2c973b172fc6-1] regionserver.HStore(310): Store=5876f55847b223c4fd0e2c973b172fc6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:43,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:43,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:43,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:43,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5876f55847b223c4fd0e2c973b172fc6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61014255, jitterRate=-0.09081675112247467}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:43,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5876f55847b223c4fd0e2c973b172fc6: 2023-11-19 23:53:43,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6., pid=55, masterSystemTime=1700438023914 2023-11-19 23:53:43,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:43,952 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=5876f55847b223c4fd0e2c973b172fc6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:43,953 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438023952"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438023952"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438023952"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438023952"}]},"ts":"1700438023952"} 2023-11-19 23:53:43,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=55, resume processing ppid=54 2023-11-19 23:53:43,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 5876f55847b223c4fd0e2c973b172fc6, server=jenkins-hbase4.apache.org,34057,1700438000374 in 192 msec 2023-11-19 23:53:43,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=54, resume processing ppid=53 2023-11-19 23:53:43,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, ASSIGN in 350 msec 2023-11-19 23:53:43,959 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:43,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438023960"}]},"ts":"1700438023960"} 2023-11-19 23:53:43,961 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-11-19 23:53:44,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:44,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 497 msec 2023-11-19 23:53:44,067 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,068 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,069 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,069 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,069 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, UNASSIGN}] 2023-11-19 23:53:44,071 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, UNASSIGN 2023-11-19 23:53:44,072 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=5876f55847b223c4fd0e2c973b172fc6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438024072"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438024072"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438024072"}]},"ts":"1700438024072"} 2023-11-19 23:53:44,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 5876f55847b223c4fd0e2c973b172fc6, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:44,225 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:44,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5876f55847b223c4fd0e2c973b172fc6, disabling compactions & flushes 2023-11-19 23:53:44,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:44,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:44,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. after waiting 0 ms 2023-11-19 23:53:44,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:44,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-11-19 23:53:44,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:44,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6. 2023-11-19 23:53:44,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5876f55847b223c4fd0e2c973b172fc6: 2023-11-19 23:53:44,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:44,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=5876f55847b223c4fd0e2c973b172fc6, regionState=CLOSED 2023-11-19 23:53:44,232 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438024232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438024232"}]},"ts":"1700438024232"} 2023-11-19 23:53:44,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=58, resume processing ppid=57 2023-11-19 23:53:44,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 5876f55847b223c4fd0e2c973b172fc6, server=jenkins-hbase4.apache.org,34057,1700438000374 in 160 msec 2023-11-19 23:53:44,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=57, resume processing ppid=56 2023-11-19 23:53:44,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5876f55847b223c4fd0e2c973b172fc6, UNASSIGN in 165 msec 2023-11-19 23:53:44,266 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-11-19 23:53:44,268 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-11-19 23:53:44,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1700438024269"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1700438024269"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1700438024269"}]},"ts":"1700438024269"} 2023-11-19 23:53:44,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024269"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438024269"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438024269"}]},"ts":"1700438024269"} 2023-11-19 23:53:44,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024269"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438024269"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438024269"}]},"ts":"1700438024269"} 2023-11-19 23:53:44,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, ASSIGN}] 2023-11-19 23:53:44,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, ASSIGN 2023-11-19 23:53:44,274 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:53:44,275 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, ASSIGN 2023-11-19 23:53:44,275 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:53:44,424 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:53:44,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=936870a1b2b873ee71f15effa88fe4b2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024426"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438024426"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438024426"}]},"ts":"1700438024426"} 2023-11-19 23:53:44,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 936870a1b2b873ee71f15effa88fe4b2, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:44,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=c06cfd0a72f03a2218b666309cabf009, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,432 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024432"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438024432"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438024432"}]},"ts":"1700438024432"} 2023-11-19 23:53:44,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure c06cfd0a72f03a2218b666309cabf009, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:44,581 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:44,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c06cfd0a72f03a2218b666309cabf009, NAME => 'testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.', STARTKEY => '', ENDKEY => '41'} 2023-11-19 23:53:44,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:44,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:44,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,586 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,588 DEBUG [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf1 2023-11-19 23:53:44,588 DEBUG [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf1 2023-11-19 23:53:44,589 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c06cfd0a72f03a2218b666309cabf009 columnFamilyName cf1 2023-11-19 23:53:44,589 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] regionserver.HStore(310): Store=c06cfd0a72f03a2218b666309cabf009/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:44,589 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,591 DEBUG [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf2 2023-11-19 23:53:44,591 DEBUG [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf2 2023-11-19 23:53:44,592 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c06cfd0a72f03a2218b666309cabf009 columnFamilyName cf2 2023-11-19 23:53:44,592 INFO [StoreOpener-c06cfd0a72f03a2218b666309cabf009-1] regionserver.HStore(310): Store=c06cfd0a72f03a2218b666309cabf009/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:44,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:44,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:44,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c06cfd0a72f03a2218b666309cabf009; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70769423, jitterRate=0.05454657971858978}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:44,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c06cfd0a72f03a2218b666309cabf009: 2023-11-19 23:53:44,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009., pid=62, masterSystemTime=1700438024581 2023-11-19 23:53:44,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:44,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:44,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:44,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 936870a1b2b873ee71f15effa88fe4b2, NAME => 'testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.', STARTKEY => '41', ENDKEY => ''} 2023-11-19 23:53:44,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=c06cfd0a72f03a2218b666309cabf009, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:44,600 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024600"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438024600"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438024600"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438024600"}]},"ts":"1700438024600"} 2023-11-19 23:53:44,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:44,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,610 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=62, resume processing ppid=59 2023-11-19 23:53:44,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure c06cfd0a72f03a2218b666309cabf009, server=jenkins-hbase4.apache.org,34057,1700438000374 in 169 msec 2023-11-19 23:53:44,612 DEBUG [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf1 2023-11-19 23:53:44,612 DEBUG [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf1 2023-11-19 23:53:44,612 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 936870a1b2b873ee71f15effa88fe4b2 columnFamilyName cf1 2023-11-19 23:53:44,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, ASSIGN in 337 msec 2023-11-19 23:53:44,613 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] regionserver.HStore(310): Store=936870a1b2b873ee71f15effa88fe4b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:44,613 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,615 DEBUG [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf2 2023-11-19 23:53:44,615 DEBUG [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf2 2023-11-19 23:53:44,615 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 936870a1b2b873ee71f15effa88fe4b2 columnFamilyName cf2 2023-11-19 23:53:44,616 INFO [StoreOpener-936870a1b2b873ee71f15effa88fe4b2-1] regionserver.HStore(310): Store=936870a1b2b873ee71f15effa88fe4b2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:44,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:44,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:44,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 936870a1b2b873ee71f15effa88fe4b2; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74816869, jitterRate=0.1148582249879837}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:44,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 936870a1b2b873ee71f15effa88fe4b2: 2023-11-19 23:53:44,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2., pid=61, masterSystemTime=1700438024581 2023-11-19 23:53:44,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:44,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:44,625 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=936870a1b2b873ee71f15effa88fe4b2, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,625 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024625"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438024625"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438024625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438024625"}]},"ts":"1700438024625"} 2023-11-19 23:53:44,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=61, resume processing ppid=60 2023-11-19 23:53:44,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 936870a1b2b873ee71f15effa88fe4b2, server=jenkins-hbase4.apache.org,34057,1700438000374 in 198 msec 2023-11-19 23:53:44,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=60, resume processing ppid=56 2023-11-19 23:53:44,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, ASSIGN in 356 msec 2023-11-19 23:53:44,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 in 564 msec 2023-11-19 23:53:44,822 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:53:44,824 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:44,826 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-11-19 23:53:44,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-11-19 23:53:44,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:44,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-11-19 23:53:44,830 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438024830"}]},"ts":"1700438024830"} 2023-11-19 23:53:44,831 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-11-19 23:53:44,919 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-11-19 23:53:44,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, UNASSIGN}] 2023-11-19 23:53:44,921 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, UNASSIGN 2023-11-19 23:53:44,922 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=936870a1b2b873ee71f15effa88fe4b2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,922 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024922"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438024922"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438024922"}]},"ts":"1700438024922"} 2023-11-19 23:53:44,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 936870a1b2b873ee71f15effa88fe4b2, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:44,925 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, UNASSIGN 2023-11-19 23:53:44,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=c06cfd0a72f03a2218b666309cabf009, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:44,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438024926"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438024926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438024926"}]},"ts":"1700438024926"} 2023-11-19 23:53:44,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure c06cfd0a72f03a2218b666309cabf009, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:44,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-11-19 23:53:45,074 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:45,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 936870a1b2b873ee71f15effa88fe4b2, disabling compactions & flushes 2023-11-19 23:53:45,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:45,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:45,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. after waiting 0 ms 2023-11-19 23:53:45,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:45,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-11-19 23:53:45,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:45,081 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2. 2023-11-19 23:53:45,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 936870a1b2b873ee71f15effa88fe4b2: 2023-11-19 23:53:45,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:45,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:45,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c06cfd0a72f03a2218b666309cabf009, disabling compactions & flushes 2023-11-19 23:53:45,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:45,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:45,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. after waiting 0 ms 2023-11-19 23:53:45,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:45,085 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=936870a1b2b873ee71f15effa88fe4b2, regionState=CLOSED 2023-11-19 23:53:45,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438025085"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438025085"}]},"ts":"1700438025085"} 2023-11-19 23:53:45,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-11-19 23:53:45,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=66, resume processing ppid=65 2023-11-19 23:53:45,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 936870a1b2b873ee71f15effa88fe4b2, server=jenkins-hbase4.apache.org,34057,1700438000374 in 164 msec 2023-11-19 23:53:45,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:45,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009. 2023-11-19 23:53:45,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c06cfd0a72f03a2218b666309cabf009: 2023-11-19 23:53:45,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=936870a1b2b873ee71f15effa88fe4b2, UNASSIGN in 169 msec 2023-11-19 23:53:45,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:45,091 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=c06cfd0a72f03a2218b666309cabf009, regionState=CLOSED 2023-11-19 23:53:45,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438025091"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438025091"}]},"ts":"1700438025091"} 2023-11-19 23:53:45,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=67, resume processing ppid=64 2023-11-19 23:53:45,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure c06cfd0a72f03a2218b666309cabf009, server=jenkins-hbase4.apache.org,34057,1700438000374 in 165 msec 2023-11-19 23:53:45,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=64, resume processing ppid=63 2023-11-19 23:53:45,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c06cfd0a72f03a2218b666309cabf009, UNASSIGN in 173 msec 2023-11-19 23:53:45,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438025096"}]},"ts":"1700438025096"} 2023-11-19 23:53:45,097 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-11-19 23:53:45,099 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-11-19 23:53:45,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 273 msec 2023-11-19 23:53:45,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-11-19 23:53:45,132 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-11-19 23:53:45,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,134 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,135 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-11-19 23:53:45,140 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:45,140 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:45,140 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:45,141 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/recovered.edits] 2023-11-19 23:53:45,142 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/recovered.edits] 2023-11-19 23:53:45,142 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/recovered.edits] 2023-11-19 23:53:45,147 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/recovered.edits/7.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2/recovered.edits/7.seqid 2023-11-19 23:53:45,147 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/recovered.edits/4.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6/recovered.edits/4.seqid 2023-11-19 23:53:45,148 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/recovered.edits/7.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009/recovered.edits/7.seqid 2023-11-19 23:53:45,148 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:53:45,148 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/5876f55847b223c4fd0e2c973b172fc6 2023-11-19 23:53:45,148 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionNoStoreFile/c06cfd0a72f03a2218b666309cabf009 2023-11-19 23:53:45,148 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-11-19 23:53:45,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,152 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-11-19 23:53:45,153 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-11-19 23:53:45,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-11-19 23:53:45,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438025154"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:45,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438025154"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:45,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438025154"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:45,156 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:53:45,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5876f55847b223c4fd0e2c973b172fc6, NAME => 'testSplitTableRegionNoStoreFile,,1700438023565.5876f55847b223c4fd0e2c973b172fc6.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c06cfd0a72f03a2218b666309cabf009, NAME => 'testSplitTableRegionNoStoreFile,,1700438024067.c06cfd0a72f03a2218b666309cabf009.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 936870a1b2b873ee71f15effa88fe4b2, NAME => 'testSplitTableRegionNoStoreFile,41,1700438024067.936870a1b2b873ee71f15effa88fe4b2.', STARTKEY => '41', ENDKEY => ''}] 2023-11-19 23:53:45,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-11-19 23:53:45,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438025156"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:45,157 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-11-19 23:53:45,160 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:53:45,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 28 msec 2023-11-19 23:53:45,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-11-19 23:53:45,236 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-11-19 23:53:45,251 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=408 (was 405) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1810756702_17 at /127.0.0.1:38404 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1810756702_17 at /127.0.0.1:38368 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=884 (was 886), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=368 (was 365) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=15196 (was 15204) 2023-11-19 23:53:45,264 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408, OpenFileDescriptor=884, MaxFileDescriptor=60000, SystemLoadAverage=368, ProcessCount=171, AvailableMemoryMB=15196 2023-11-19 23:53:45,264 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:45,266 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:45,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:45,268 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,269 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:45,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:45,271 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,272 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b empty. 2023-11-19 23:53:45,272 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,272 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-11-19 23:53:45,286 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:45,288 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7733ca898e07f4f307912b72a6278a2b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing 7733ca898e07f4f307912b72a6278a2b, disabling compactions & flushes 2023-11-19 23:53:45,296 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. after waiting 0 ms 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,296 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,296 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:45,298 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:45,299 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438025299"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438025299"}]},"ts":"1700438025299"} 2023-11-19 23:53:45,300 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:45,301 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:45,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438025301"}]},"ts":"1700438025301"} 2023-11-19 23:53:45,302 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-11-19 23:53:45,305 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:45,306 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:45,306 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:45,306 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:45,306 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:45,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, ASSIGN}] 2023-11-19 23:53:45,308 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, ASSIGN 2023-11-19 23:53:45,308 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:45,459 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:45,460 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7733ca898e07f4f307912b72a6278a2b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,460 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438025460"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438025460"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438025460"}]},"ts":"1700438025460"} 2023-11-19 23:53:45,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 7733ca898e07f4f307912b72a6278a2b, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:45,613 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7733ca898e07f4f307912b72a6278a2b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:45,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:45,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:45,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,621 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,623 DEBUG [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1 2023-11-19 23:53:45,623 DEBUG [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1 2023-11-19 23:53:45,623 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7733ca898e07f4f307912b72a6278a2b columnFamilyName cf1 2023-11-19 23:53:45,624 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] regionserver.HStore(310): Store=7733ca898e07f4f307912b72a6278a2b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:45,624 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,626 DEBUG [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2 2023-11-19 23:53:45,626 DEBUG [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2 2023-11-19 23:53:45,626 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7733ca898e07f4f307912b72a6278a2b columnFamilyName cf2 2023-11-19 23:53:45,627 INFO [StoreOpener-7733ca898e07f4f307912b72a6278a2b-1] regionserver.HStore(310): Store=7733ca898e07f4f307912b72a6278a2b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:45,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:45,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:45,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:45,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7733ca898e07f4f307912b72a6278a2b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58909442, jitterRate=-0.12218090891838074}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:45,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:45,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b., pid=71, masterSystemTime=1700438025613 2023-11-19 23:53:45,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7733ca898e07f4f307912b72a6278a2b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438025638"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438025638"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438025638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438025638"}]},"ts":"1700438025638"} 2023-11-19 23:53:45,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=71, resume processing ppid=70 2023-11-19 23:53:45,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 7733ca898e07f4f307912b72a6278a2b, server=jenkins-hbase4.apache.org,39613,1700438000411 in 177 msec 2023-11-19 23:53:45,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=70, resume processing ppid=69 2023-11-19 23:53:45,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, ASSIGN in 334 msec 2023-11-19 23:53:45,644 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:45,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438025644"}]},"ts":"1700438025644"} 2023-11-19 23:53:45,645 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-11-19 23:53:45,648 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:45,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 381 msec 2023-11-19 23:53:45,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:45,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,778 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:45,778 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,778 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:45,778 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:45,778 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,778 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,780 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,780 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,780 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,780 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,781 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,781 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,782 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:45,783 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:45,783 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,783 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:45,784 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:53:45,785 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,785 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,786 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,786 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,786 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:45,786 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:45,796 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/9c6c5b867a77484683c0e5f7bd1912a1 2023-11-19 23:53:45,807 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 2023-11-19 23:53:45,811 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/9c6c5b867a77484683c0e5f7bd1912a1 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1 2023-11-19 23:53:45,815 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:45,816 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 2023-11-19 23:53:45,820 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:53:45,821 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 37ms, sequenceid=6, compaction requested=false 2023-11-19 23:53:45,821 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:45,821 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:45,821 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:45,821 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:45,834 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,834 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:45,834 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,834 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:45,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:45,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:45,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@732bfbf0[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:45,836 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:45,836 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:45,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,838 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,838 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:45,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:45,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:45,841 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:45,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:45,845 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:45,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:45,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:45,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,845 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:45,848 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,848 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,849 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:45,850 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,851 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,851 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,856 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,855 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:45,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:45,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:45,856 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,857 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:45,857 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,957 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:45,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:45,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:45,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:45,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,974 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:45,974 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,974 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:45,974 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:45,974 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,974 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,976 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,976 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,976 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:45,976 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,978 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:45,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,978 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:45,979 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:45,979 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:45,979 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:45,979 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:45,979 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:45,979 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:45,979 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:45,983 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,984 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,984 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,984 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:45,985 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:45,985 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:45,990 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/6a2cd21c82f54733be044f68ffc15eb9 2023-11-19 23:53:46,001 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc 2023-11-19 23:53:46,005 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/6a2cd21c82f54733be044f68ffc15eb9 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9 2023-11-19 23:53:46,009 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:46,010 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc 2023-11-19 23:53:46,013 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:53:46,014 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 35ms, sequenceid=19, compaction requested=false 2023-11-19 23:53:46,014 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,014 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,014 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,014 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,014 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,014 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,014 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,016 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2232e4cc[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,018 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,019 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,020 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,021 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,021 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,023 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,027 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,027 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,029 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,030 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,031 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,032 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,033 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,033 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,036 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,036 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,037 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,137 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:46,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:46,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:46,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:46,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,152 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:46,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:46,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,152 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,154 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,154 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,154 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:46,156 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,156 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,156 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:46,159 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,159 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,159 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,159 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,160 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,160 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,170 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/cd93de537f4c46948e31abe21427deeb 2023-11-19 23:53:46,181 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 2023-11-19 23:53:46,185 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/cd93de537f4c46948e31abe21427deeb as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb 2023-11-19 23:53:46,189 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:46,190 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 2023-11-19 23:53:46,194 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:53:46,195 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 39ms, sequenceid=32, compaction requested=true 2023-11-19 23:53:46,195 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,195 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,197 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,197 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,198 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,198 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47e77b27[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,200 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,200 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,201 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,201 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,201 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,201 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,201 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,202 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,202 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,206 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,210 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,210 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,212 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,213 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,213 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,214 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,215 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,216 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,216 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,220 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,219 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,219 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,220 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,220 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,221 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,221 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,221 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,320 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:46,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:46,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:46,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:46,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,335 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,335 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,335 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:46,335 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:46,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,338 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,339 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,339 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,340 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:46,341 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,341 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,341 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:46,341 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,343 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,343 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,343 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,350 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/8964da69ac324f33a4579d0d8e2fb83c 2023-11-19 23:53:46,361 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/a104ba77f79a41dd92074a4731824c12 2023-11-19 23:53:46,365 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/8964da69ac324f33a4579d0d8e2fb83c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c 2023-11-19 23:53:46,368 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:46,369 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/a104ba77f79a41dd92074a4731824c12 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12 2023-11-19 23:53:46,372 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:53:46,373 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 32ms, sequenceid=45, compaction requested=true 2023-11-19 23:53:46,373 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,373 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,375 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,375 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5182bd11[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,377 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,377 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,379 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,379 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,379 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,380 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,381 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,382 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,385 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,385 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,385 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,385 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,385 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,387 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,388 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,388 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,388 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,390 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,390 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,391 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,397 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,397 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,498 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:46,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:46,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:46,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,511 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:46,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:46,512 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,512 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,514 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,514 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,515 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,515 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,516 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,516 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:46,516 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,516 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,516 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:46,517 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,517 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,518 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,518 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,518 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,518 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,525 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/8642eb2f17a9462aa14b168d29a13dd8 2023-11-19 23:53:46,536 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/c0720ce7632e453f9c26d498fb4ce6aa 2023-11-19 23:53:46,540 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/8642eb2f17a9462aa14b168d29a13dd8 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8 2023-11-19 23:53:46,544 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:46,545 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/c0720ce7632e453f9c26d498fb4ce6aa as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa 2023-11-19 23:53:46,549 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:53:46,550 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 34ms, sequenceid=58, compaction requested=true 2023-11-19 23:53:46,550 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,550 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,550 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,550 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,550 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,550 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,550 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,552 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,552 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,552 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,553 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dda28aa[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,556 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,556 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,557 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,557 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,557 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,558 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,558 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,558 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,558 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,558 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,558 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,561 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,561 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,562 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,567 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,567 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,567 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,567 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,567 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,570 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,571 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,571 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,571 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,573 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,574 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,574 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,577 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,576 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,577 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,578 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,578 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,578 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,578 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,578 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,678 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:46,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:46,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:46,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:46,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,694 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,694 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,694 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:46,694 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:46,694 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,694 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,696 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,696 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,696 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,698 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,699 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:46,699 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:46,699 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,699 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:53:46,701 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,705 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,706 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,706 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,706 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,709 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/e8497cb94e1e4cce955a86a981bd76cd 2023-11-19 23:53:46,723 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/29bd3d11a3944b7281eff4bf7f719e54 2023-11-19 23:53:46,727 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/e8497cb94e1e4cce955a86a981bd76cd as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd 2023-11-19 23:53:46,731 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:46,732 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/29bd3d11a3944b7281eff4bf7f719e54 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54 2023-11-19 23:53:46,738 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:53:46,739 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 40ms, sequenceid=71, compaction requested=true 2023-11-19 23:53:46,739 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,739 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,739 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,739 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,739 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,739 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,739 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,741 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,741 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,741 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,742 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@487bb759[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,744 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,744 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,746 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,746 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,746 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,749 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,750 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,750 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,753 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,753 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,755 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,756 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,756 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,757 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,758 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,759 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,762 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,763 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,864 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:46,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:46,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:46,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:46,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,879 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,879 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,879 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:46,879 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:46,879 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,879 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,880 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,880 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,880 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,880 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,881 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,881 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,882 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:46,882 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,882 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,882 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:46,883 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,883 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:46,883 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-11-19 23:53:46,884 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,886 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,886 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,886 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,886 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,886 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,896 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:46,900 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:46,909 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:46,912 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:46,913 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/c30edc8a799e4416931edcdb2b9201a3 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:46,916 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:46,916 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3, entries=9, sequenceid=83, filesize=5.1 K 2023-11-19 23:53:46,917 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/7ffa31ca494f4264b6764ab0db097f41 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:46,920 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:46,921 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41, entries=9, sequenceid=83, filesize=5.1 K 2023-11-19 23:53:46,921 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 38ms, sequenceid=83, compaction requested=true 2023-11-19 23:53:46,921 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:46,921 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:46,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:46,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:46,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,921 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:46,922 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:46,923 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,923 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,923 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,923 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:46,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:46,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3439d332[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:46,925 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:46,925 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:46,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,927 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:46,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,929 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,929 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,929 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,938 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:46,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:46,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:46,941 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:46,941 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:46,941 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:46,941 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,941 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:46,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,943 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,943 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,944 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,944 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:46,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,946 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,946 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,946 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,949 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:46,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:46,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:46,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:46,949 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:46,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:46,950 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,050 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:47,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:47,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:47,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:47,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,060 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:47,060 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,060 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:47,060 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:47,061 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,061 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,063 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,063 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,063 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,063 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,063 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:47,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,064 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,065 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,065 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:47,065 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:47,065 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,066 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:47,066 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,076 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,080 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,087 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,090 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,090 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,094 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,094 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0, entries=5, sequenceid=91, filesize=5.0 K 2023-11-19 23:53:47,094 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/2c0f90b1ccc54e7aa2a39476c84669bf as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,098 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,098 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf, entries=5, sequenceid=91, filesize=5.0 K 2023-11-19 23:53:47,099 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 33ms, sequenceid=91, compaction requested=true 2023-11-19 23:53:47,099 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:47,099 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,101 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,101 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,101 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,101 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:47,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70979a91[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:47,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:47,103 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,104 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,104 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:47,104 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:47,107 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,107 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,108 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:47,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:47,110 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:47,111 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:47,111 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:47,111 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,111 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,112 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,113 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,114 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,115 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,116 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,116 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:47,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,122 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:47,122 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:47,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,123 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,223 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:47,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:47,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:53:47,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:53:47,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,233 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:47,233 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,233 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-11-19 23:53:47,233 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-11-19 23:53:47,233 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,233 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,235 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,235 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,235 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,235 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-11-19 23:53:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:53:47,238 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,238 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. started... 2023-11-19 23:53:47,238 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-11-19 23:53:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,255 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,258 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,278 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,282 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,283 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/0ce42bff9af84030a2a2fb64d3e4c48e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,287 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,287 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e, entries=5, sequenceid=99, filesize=5.0 K 2023-11-19 23:53:47,288 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/d7a46029b90b4f828c6b81c47d896197 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,292 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,292 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197, entries=5, sequenceid=99, filesize=5.0 K 2023-11-19 23:53:47,293 INFO [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 55ms, sequenceid=99, compaction requested=true 2023-11-19 23:53:47,293 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:47,293 DEBUG [rs(jenkins-hbase4.apache.org,39613,1700438000411)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-11-19 23:53:47,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-11-19 23:53:47,295 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,295 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-11-19 23:53:47,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ca47351[Count = 0] remaining members to acquire global barrier 2023-11-19 23:53:47,297 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-11-19 23:53:47,297 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,299 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,299 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:53:47,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:53:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,301 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-11-19 23:53:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39613,1700438000411': 2023-11-19 23:53:47,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:53:47,304 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-11-19 23:53:47,304 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:53:47,304 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:53:47,304 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,304 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,308 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,309 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:53:47,309 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,309 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:53:47,311 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,312 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,312 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,314 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:53:47,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-11-19 23:53:47,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:47,315 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:53:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:53:47,315 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:53:47,316 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:47,316 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:53:47,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:53:47,416 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:53:47,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:53:47,419 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,419 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:47,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:47,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:47,420 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:47,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, UNASSIGN}] 2023-11-19 23:53:47,423 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, UNASSIGN 2023-11-19 23:53:47,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7733ca898e07f4f307912b72a6278a2b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438027423"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438027423"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438027423"}]},"ts":"1700438027423"} 2023-11-19 23:53:47,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 7733ca898e07f4f307912b72a6278a2b, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:47,575 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:47,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7733ca898e07f4f307912b72a6278a2b, disabling compactions & flushes 2023-11-19 23:53:47,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. after waiting 0 ms 2023-11-19 23:53:47,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7733ca898e07f4f307912b72a6278a2b 2/2 column families, dataSize=50 B heapSize=720 B 2023-11-19 23:53:47,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,589 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf1/63926b3f08874124b4dd6e937d54a044 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044, entries=1, sequenceid=103, filesize=4.9 K 2023-11-19 23:53:47,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.tmp/cf2/51cfd63863cc4a47bcbd0f19f5732758 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758, entries=1, sequenceid=103, filesize=4.9 K 2023-11-19 23:53:47,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 7733ca898e07f4f307912b72a6278a2b in 33ms, sequenceid=103, compaction requested=true 2023-11-19 23:53:47,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-11-19 23:53:47,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:47,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b. 2023-11-19 23:53:47,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7733ca898e07f4f307912b72a6278a2b: 2023-11-19 23:53:47,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7733ca898e07f4f307912b72a6278a2b, regionState=CLOSED 2023-11-19 23:53:47,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438027616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438027616"}]},"ts":"1700438027616"} 2023-11-19 23:53:47,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=74, resume processing ppid=73 2023-11-19 23:53:47,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 7733ca898e07f4f307912b72a6278a2b, server=jenkins-hbase4.apache.org,39613,1700438000411 in 193 msec 2023-11-19 23:53:47,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=73, resume processing ppid=72 2023-11-19 23:53:47,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7733ca898e07f4f307912b72a6278a2b, UNASSIGN in 197 msec 2023-11-19 23:53:47,632 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=7733ca898e07f4f307912b72a6278a2b, threads=16 2023-11-19 23:53:47,632 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,632 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,640 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,640 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,640 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,641 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,641 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,641 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,645 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,645 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,646 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,646 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,646 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,646 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,646 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,648 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,651 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12, top=false 2023-11-19 23:53:47,653 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54, top=false 2023-11-19 23:53:47,656 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,657 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:47,659 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,661 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,661 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c, top=false 2023-11-19 23:53:47,661 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8, top=false 2023-11-19 23:53:47,665 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa, top=false 2023-11-19 23:53:47,666 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,666 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,670 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,671 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:47,671 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044, top=true 2023-11-19 23:53:47,672 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:47,673 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758, top=true 2023-11-19 23:53:47,675 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:47,675 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf, top=true 2023-11-19 23:53:47,677 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:47,677 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0, top=true 2023-11-19 23:53:47,684 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:47,684 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197, top=true 2023-11-19 23:53:47,698 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8 for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,698 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:47,698 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e, top=true 2023-11-19 23:53:47,699 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12 for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,703 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,703 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,703 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,704 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54 for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,705 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,705 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,705 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,710 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197 for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,710 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,710 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,714 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,714 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,714 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0 for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,715 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,715 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,715 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,716 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:47,717 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,717 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044 for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,717 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758 for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,717 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,717 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,720 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,726 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e for child: 2c794c96882fdb1d29833ea1af058f2f, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,726 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,726 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd, top=false 2023-11-19 23:53:47,754 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/.splits/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd for child: a9c85c60733c72c1431e0d5b15931758, parent: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,757 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,771 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:47,771 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,788 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,792 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,794 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,800 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,804 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:47,807 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,112 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:48,118 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41 for region: 7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,118 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 7733ca898e07f4f307912b72a6278a2b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:53:48,158 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-11-19 23:53:48,160 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-11-19 23:53:48,161 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1700438028161"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1700438028161"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1700438028161"}]},"ts":"1700438028161"} 2023-11-19 23:53:48,161 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028161"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438028161"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438028161"}]},"ts":"1700438028161"} 2023-11-19 23:53:48,161 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028161"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438028161"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438028161"}]},"ts":"1700438028161"} 2023-11-19 23:53:48,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, ASSIGN}] 2023-11-19 23:53:48,164 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, ASSIGN 2023-11-19 23:53:48,165 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:48,166 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, ASSIGN 2023-11-19 23:53:48,166 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39613,1700438000411; forceNewPlan=false, retain=false 2023-11-19 23:53:48,315 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:53:48,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=a9c85c60733c72c1431e0d5b15931758, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028317"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438028317"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438028317"}]},"ts":"1700438028317"} 2023-11-19 23:53:48,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure a9c85c60733c72c1431e0d5b15931758, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:48,321 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2c794c96882fdb1d29833ea1af058f2f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,321 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028321"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438028321"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438028321"}]},"ts":"1700438028321"} 2023-11-19 23:53:48,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 2c794c96882fdb1d29833ea1af058f2f, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:48,438 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:53:48,470 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c794c96882fdb1d29833ea1af058f2f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.', STARTKEY => '60', ENDKEY => ''} 2023-11-19 23:53:48,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,474 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,475 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1 2023-11-19 23:53:48,475 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1 2023-11-19 23:53:48,476 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c794c96882fdb1d29833ea1af058f2f columnFamilyName cf1 2023-11-19 23:53:48,486 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-top 2023-11-19 23:53:48,491 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-top 2023-11-19 23:53:48,494 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,495 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-top 2023-11-19 23:53:48,499 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-top 2023-11-19 23:53:48,502 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:48,502 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:48,505 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:48,505 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:48,509 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:48,509 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:48,509 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(310): Store=2c794c96882fdb1d29833ea1af058f2f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:48,509 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,510 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2 2023-11-19 23:53:48,510 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2 2023-11-19 23:53:48,511 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c794c96882fdb1d29833ea1af058f2f columnFamilyName cf2 2023-11-19 23:53:48,520 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,521 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-top 2023-11-19 23:53:48,525 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-top 2023-11-19 23:53:48,529 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-top 2023-11-19 23:53:48,533 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-top 2023-11-19 23:53:48,536 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:48,536 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:48,539 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:48,539 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:48,542 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:48,542 DEBUG [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:48,542 INFO [StoreOpener-2c794c96882fdb1d29833ea1af058f2f-1] regionserver.HStore(310): Store=2c794c96882fdb1d29833ea1af058f2f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:48,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:48,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:48,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2c794c96882fdb1d29833ea1af058f2f; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73704276, jitterRate=0.09827929735183716}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:48,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:48,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f., pid=78, masterSystemTime=1700438028470 2023-11-19 23:53:48,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,549 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:48,549 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,549 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1912): 2c794c96882fdb1d29833ea1af058f2f/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,549 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1912): 2c794c96882fdb1d29833ea1af058f2f/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c794c96882fdb1d29833ea1af058f2f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c794c96882fdb1d29833ea1af058f2f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp, totalSize=34.8 K 2023-11-19 23:53:48,549 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp, totalSize=34.8 K 2023-11-19 23:53:48,550 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438025771 2023-11-19 23:53:48,550 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438025771 2023-11-19 23:53:48,550 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438025959 2023-11-19 23:53:48,550 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438025959 2023-11-19 23:53:48,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a9c85c60733c72c1431e0d5b15931758, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.', STARTKEY => '', ENDKEY => '60'} 2023-11-19 23:53:48,551 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438026139 2023-11-19 23:53:48,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2c794c96882fdb1d29833ea1af058f2f, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028551"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438028551"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438028551"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438028551"}]},"ts":"1700438028551"} 2023-11-19 23:53:48,551 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1700438026865 2023-11-19 23:53:48,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:48,551 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438026139 2023-11-19 23:53:48,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:48,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,551 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,552 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1700438026865 2023-11-19 23:53:48,552 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,552 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,553 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,553 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,553 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,553 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,554 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1 2023-11-19 23:53:48,554 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1 2023-11-19 23:53:48,554 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9c85c60733c72c1431e0d5b15931758 columnFamilyName cf1 2023-11-19 23:53:48,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=78, resume processing ppid=76 2023-11-19 23:53:48,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 2c794c96882fdb1d29833ea1af058f2f, server=jenkins-hbase4.apache.org,39613,1700438000411 in 231 msec 2023-11-19 23:53:48,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, ASSIGN in 391 msec 2023-11-19 23:53:48,565 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-bottom 2023-11-19 23:53:48,568 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-bottom 2023-11-19 23:53:48,573 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,574 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-bottom 2023-11-19 23:53:48,574 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2c794c96882fdb1d29833ea1af058f2f#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,574 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): 2c794c96882fdb1d29833ea1af058f2f#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,579 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-bottom 2023-11-19 23:53:48,582 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8 2023-11-19 23:53:48,590 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c 2023-11-19 23:53:48,594 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd 2023-11-19 23:53:48,596 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(310): Store=a9c85c60733c72c1431e0d5b15931758/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:48,596 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,597 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fbe3ac84dfec48168de6bfeea0151511: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,597 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2 2023-11-19 23:53:48,597 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2 2023-11-19 23:53:48,598 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9c85c60733c72c1431e0d5b15931758 columnFamilyName cf2 2023-11-19 23:53:48,598 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp/cf1/fbe3ac84dfec48168de6bfeea0151511 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/fbe3ac84dfec48168de6bfeea0151511 2023-11-19 23:53:48,602 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f509c155a9c64114b7736a11b67202e5: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,602 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fbe3ac84dfec48168de6bfeea0151511: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,603 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp/cf2/f509c155a9c64114b7736a11b67202e5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f509c155a9c64114b7736a11b67202e5 2023-11-19 23:53:48,603 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2c794c96882fdb1d29833ea1af058f2f/cf1 of 2c794c96882fdb1d29833ea1af058f2f into fbe3ac84dfec48168de6bfeea0151511(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,603 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:48,603 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f., storeName=2c794c96882fdb1d29833ea1af058f2f/cf1, priority=9, startTime=1700438028548; duration=0sec 2023-11-19 23:53:48,603 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,606 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f509c155a9c64114b7736a11b67202e5: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,607 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:48,607 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2c794c96882fdb1d29833ea1af058f2f/cf2 of 2c794c96882fdb1d29833ea1af058f2f into f509c155a9c64114b7736a11b67202e5(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,607 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-bottom 2023-11-19 23:53:48,607 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:48,607 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f., storeName=2c794c96882fdb1d29833ea1af058f2f/cf2, priority=9, startTime=1700438028549; duration=0sec 2023-11-19 23:53:48,607 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,610 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-bottom 2023-11-19 23:53:48,613 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-bottom 2023-11-19 23:53:48,616 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-bottom 2023-11-19 23:53:48,620 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54 2023-11-19 23:53:48,623 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12 2023-11-19 23:53:48,626 DEBUG [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa 2023-11-19 23:53:48,626 INFO [StoreOpener-a9c85c60733c72c1431e0d5b15931758-1] regionserver.HStore(310): Store=a9c85c60733c72c1431e0d5b15931758/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:48,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:53:48,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:48,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a9c85c60733c72c1431e0d5b15931758; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63892860, jitterRate=-0.04792219400405884}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:53:48,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a9c85c60733c72c1431e0d5b15931758: 2023-11-19 23:53:48,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758., pid=77, masterSystemTime=1700438028470 2023-11-19 23:53:48,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,633 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:48,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,633 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,634 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1912): a9c85c60733c72c1431e0d5b15931758/cf1 is initiating minor compaction (all files) 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,634 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1912): a9c85c60733c72c1431e0d5b15931758/cf2 is initiating minor compaction (all files) 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a9c85c60733c72c1431e0d5b15931758/cf1 in testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a9c85c60733c72c1431e0d5b15931758/cf2 in testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp, totalSize=35.0 K 2023-11-19 23:53:48,634 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp, totalSize=35.0 K 2023-11-19 23:53:48,634 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438025771 2023-11-19 23:53:48,635 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438025771 2023-11-19 23:53:48,635 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438025959 2023-11-19 23:53:48,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,635 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438025959 2023-11-19 23:53:48,635 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438026139 2023-11-19 23:53:48,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=a9c85c60733c72c1431e0d5b15931758, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,636 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438026139 2023-11-19 23:53:48,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028635"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438028635"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438028635"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438028635"}]},"ts":"1700438028635"} 2023-11-19 23:53:48,636 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438026322 2023-11-19 23:53:48,636 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438026322 2023-11-19 23:53:48,636 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438026499 2023-11-19 23:53:48,637 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438026499 2023-11-19 23:53:48,637 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438026679 2023-11-19 23:53:48,637 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438026679 2023-11-19 23:53:48,638 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438026865 2023-11-19 23:53:48,638 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting 7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438026865 2023-11-19 23:53:48,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=77, resume processing ppid=75 2023-11-19 23:53:48,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure a9c85c60733c72c1431e0d5b15931758, server=jenkins-hbase4.apache.org,39613,1700438000411 in 319 msec 2023-11-19 23:53:48,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=75, resume processing ppid=72 2023-11-19 23:53:48,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, ASSIGN in 475 msec 2023-11-19 23:53:48,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f in 1.2230 sec 2023-11-19 23:53:48,652 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9c85c60733c72c1431e0d5b15931758#cf1#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,653 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): a9c85c60733c72c1431e0d5b15931758#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,682 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp/cf2/866684d3e7da4366a63f14630018cd35 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/866684d3e7da4366a63f14630018cd35 2023-11-19 23:53:48,683 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp/cf1/26598a6797c74670b5045ee43fb2def2 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/26598a6797c74670b5045ee43fb2def2 2023-11-19 23:53:48,684 DEBUG [Listener at localhost/42385] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-19 23:53:48,690 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51770, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-19 23:53:48,690 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,691 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a9c85c60733c72c1431e0d5b15931758/cf1 of a9c85c60733c72c1431e0d5b15931758 into 26598a6797c74670b5045ee43fb2def2(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,691 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a9c85c60733c72c1431e0d5b15931758: 2023-11-19 23:53:48,691 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758., storeName=a9c85c60733c72c1431e0d5b15931758/cf1, priority=9, startTime=1700438028632; duration=0sec 2023-11-19 23:53:48,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-11-19 23:53:48,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.HStore(1912): a9c85c60733c72c1431e0d5b15931758/cf1 is initiating major compaction (all files) 2023-11-19 23:53:48,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@21c808f2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,692 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-11-19 23:53:48,692 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-19 23:53:48,692 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-19 23:53:48,692 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. because compaction request was cancelled 2023-11-19 23:53:48,692 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,692 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a9c85c60733c72c1431e0d5b15931758/cf1 in testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:48,692 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/26598a6797c74670b5045ee43fb2def2] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp, totalSize=6.8 K 2023-11-19 23:53:48,693 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] compactions.Compactor(207): Compacting 26598a6797c74670b5045ee43fb2def2, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438025771 2023-11-19 23:53:48,693 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a9c85c60733c72c1431e0d5b15931758/cf2 of a9c85c60733c72c1431e0d5b15931758 into 866684d3e7da4366a63f14630018cd35(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,693 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a9c85c60733c72c1431e0d5b15931758: 2023-11-19 23:53:48,693 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758., storeName=a9c85c60733c72c1431e0d5b15931758/cf2, priority=9, startTime=1700438028633; duration=0sec 2023-11-19 23:53:48,693 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,694 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.HStore(1912): 2c794c96882fdb1d29833ea1af058f2f/cf1 is initiating major compaction (all files) 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3c0958d7; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,694 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c794c96882fdb1d29833ea1af058f2f/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-11-19 23:53:48,694 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/fbe3ac84dfec48168de6bfeea0151511] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp, totalSize=5.0 K 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.HStore(1912): 2c794c96882fdb1d29833ea1af058f2f/cf2 is initiating major compaction (all files) 2023-11-19 23:53:48,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6ec7445; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,695 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting fbe3ac84dfec48168de6bfeea0151511, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,695 INFO [Listener at localhost/42385] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-11-19 23:53:48,697 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9c85c60733c72c1431e0d5b15931758#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,698 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): 2c794c96882fdb1d29833ea1af058f2f#cf1#compaction#77 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,713 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. is not mergeable because it has references 2023-11-19 23:53:48,717 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/.tmp/cf1/ef4b5b73e2664c99b41b76402f85258e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/ef4b5b73e2664c99b41b76402f85258e 2023-11-19 23:53:48,717 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3a64028de32f4e8b8614d29dfb710a9f: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,718 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp/cf1/3a64028de32f4e8b8614d29dfb710a9f as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/3a64028de32f4e8b8614d29dfb710a9f 2023-11-19 23:53:48,720 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. is not mergeable because it has references 2023-11-19 23:53:48,722 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in a9c85c60733c72c1431e0d5b15931758/cf1 of a9c85c60733c72c1431e0d5b15931758 into ef4b5b73e2664c99b41b76402f85258e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,722 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a9c85c60733c72c1431e0d5b15931758: 2023-11-19 23:53:48,723 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3a64028de32f4e8b8614d29dfb710a9f: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,724 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2c794c96882fdb1d29833ea1af058f2f/cf1 of 2c794c96882fdb1d29833ea1af058f2f into 3a64028de32f4e8b8614d29dfb710a9f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,724 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:48,724 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1700438028694; duration=0sec 2023-11-19 23:53:48,724 INFO [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1700438028691; duration=0sec 2023-11-19 23:53:48,724 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:53:48,724 DEBUG [RS:2;jenkins-hbase4:39613-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,724 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c794c96882fdb1d29833ea1af058f2f/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:48,724 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f509c155a9c64114b7736a11b67202e5] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp, totalSize=5.0 K 2023-11-19 23:53:48,726 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] compactions.Compactor(207): Compacting f509c155a9c64114b7736a11b67202e5, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-11-19 23:53:48,728 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] throttle.PressureAwareThroughputController(145): 2c794c96882fdb1d29833ea1af058f2f#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:53:48,742 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 760b2dbf36984af0864ac433c40afa53: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,743 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/.tmp/cf2/760b2dbf36984af0864ac433c40afa53 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/760b2dbf36984af0864ac433c40afa53 2023-11-19 23:53:48,747 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 760b2dbf36984af0864ac433c40afa53: NONE, but ROW specified in column family configuration 2023-11-19 23:53:48,748 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2c794c96882fdb1d29833ea1af058f2f/cf2 of 2c794c96882fdb1d29833ea1af058f2f into 760b2dbf36984af0864ac433c40afa53(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:53:48,748 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:48,748 INFO [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1700438028694; duration=0sec 2023-11-19 23:53:48,748 DEBUG [RS:2;jenkins-hbase4:39613-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:53:48,829 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39613] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. is not mergeable because it has references 2023-11-19 23:53:48,829 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39613] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. is not mergeable because it has references 2023-11-19 23:53:48,836 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-11-19 23:53:48,837 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:48,839 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:48,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:48,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:48,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-19 23:53:48,843 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438028843"}]},"ts":"1700438028843"} 2023-11-19 23:53:48,844 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-11-19 23:53:48,846 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-11-19 23:53:48,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, UNASSIGN}] 2023-11-19 23:53:48,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, UNASSIGN 2023-11-19 23:53:48,847 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2c794c96882fdb1d29833ea1af058f2f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,848 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028847"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438028847"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438028847"}]},"ts":"1700438028847"} 2023-11-19 23:53:48,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 2c794c96882fdb1d29833ea1af058f2f, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:48,851 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, UNASSIGN 2023-11-19 23:53:48,851 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a9c85c60733c72c1431e0d5b15931758, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:48,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438028851"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438028851"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438028851"}]},"ts":"1700438028851"} 2023-11-19 23:53:48,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure a9c85c60733c72c1431e0d5b15931758, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:48,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-19 23:53:49,000 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:49,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:49,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a9c85c60733c72c1431e0d5b15931758, disabling compactions & flushes 2023-11-19 23:53:49,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:49,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:49,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. after waiting 0 ms 2023-11-19 23:53:49,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:49,001 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/26598a6797c74670b5045ee43fb2def2, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-bottom] to archive 2023-11-19 23:53:49,002 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:49,004 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,005 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,006 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,007 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8964da69ac324f33a4579d0d8e2fb83c 2023-11-19 23:53:49,008 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-8642eb2f17a9462aa14b168d29a13dd8 2023-11-19 23:53:49,009 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-e8497cb94e1e4cce955a86a981bd76cd 2023-11-19 23:53:49,010 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/26598a6797c74670b5045ee43fb2def2 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/26598a6797c74670b5045ee43fb2def2 2023-11-19 23:53:49,011 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,012 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-bottom] to archive 2023-11-19 23:53:49,013 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:49,014 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,015 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,016 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,017 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-a104ba77f79a41dd92074a4731824c12 2023-11-19 23:53:49,018 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-c0720ce7632e453f9c26d498fb4ce6aa 2023-11-19 23:53:49,019 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-29bd3d11a3944b7281eff4bf7f719e54 2023-11-19 23:53:49,020 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/recovered.edits/112.seqid, newMaxSeqId=112, maxSeqId=106 2023-11-19 23:53:49,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:49,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758. 2023-11-19 23:53:49,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a9c85c60733c72c1431e0d5b15931758: 2023-11-19 23:53:49,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:49,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:49,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2c794c96882fdb1d29833ea1af058f2f, disabling compactions & flushes 2023-11-19 23:53:49,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:49,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:49,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. after waiting 0 ms 2023-11-19 23:53:49,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:49,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=a9c85c60733c72c1431e0d5b15931758, regionState=CLOSED 2023-11-19 23:53:49,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438029025"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438029025"}]},"ts":"1700438029025"} 2023-11-19 23:53:49,026 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/fbe3ac84dfec48168de6bfeea0151511, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044] to archive 2023-11-19 23:53:49,027 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:49,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=83, resume processing ppid=80 2023-11-19 23:53:49,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure a9c85c60733c72c1431e0d5b15931758, server=jenkins-hbase4.apache.org,39613,1700438000411 in 175 msec 2023-11-19 23:53:49,029 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/9c6c5b867a77484683c0e5f7bd1912a1.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a9c85c60733c72c1431e0d5b15931758, UNASSIGN in 182 msec 2023-11-19 23:53:49,030 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/6a2cd21c82f54733be044f68ffc15eb9.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,031 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/cd93de537f4c46948e31abe21427deeb.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,032 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/c30edc8a799e4416931edcdb2b9201a3.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,033 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:49,033 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:49,034 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/fbe3ac84dfec48168de6bfeea0151511 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/fbe3ac84dfec48168de6bfeea0151511 2023-11-19 23:53:49,035 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:49,037 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f509c155a9c64114b7736a11b67202e5, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758] to archive 2023-11-19 23:53:49,037 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:53:49,038 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,039 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,040 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f34788f58c1b4dfab221c9fd4c1e73e6.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,041 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/7ffa31ca494f4264b6764ab0db097f41.7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,042 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:49,043 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:49,044 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f509c155a9c64114b7736a11b67202e5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/f509c155a9c64114b7736a11b67202e5 2023-11-19 23:53:49,044 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/testSplitTableRegionDeletedRowsDaughter=7733ca898e07f4f307912b72a6278a2b-51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:49,047 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-11-19 23:53:49,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:49,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f. 2023-11-19 23:53:49,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2c794c96882fdb1d29833ea1af058f2f: 2023-11-19 23:53:49,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:49,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2c794c96882fdb1d29833ea1af058f2f, regionState=CLOSED 2023-11-19 23:53:49,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1700438029049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438029049"}]},"ts":"1700438029049"} 2023-11-19 23:53:49,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=82, resume processing ppid=81 2023-11-19 23:53:49,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 2c794c96882fdb1d29833ea1af058f2f, server=jenkins-hbase4.apache.org,39613,1700438000411 in 201 msec 2023-11-19 23:53:49,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=81, resume processing ppid=79 2023-11-19 23:53:49,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2c794c96882fdb1d29833ea1af058f2f, UNASSIGN in 206 msec 2023-11-19 23:53:49,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438029054"}]},"ts":"1700438029054"} 2023-11-19 23:53:49,054 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-11-19 23:53:49,056 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-11-19 23:53:49,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 217 msec 2023-11-19 23:53:49,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-19 23:53:49,144 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-11-19 23:53:49,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,147 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,147 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-11-19 23:53:49,151 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,151 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:49,151 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:49,153 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/recovered.edits] 2023-11-19 23:53:49,153 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/recovered.edits] 2023-11-19 23:53:49,154 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/recovered.edits] 2023-11-19 23:53:49,156 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/3a64028de32f4e8b8614d29dfb710a9f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf1/3a64028de32f4e8b8614d29dfb710a9f 2023-11-19 23:53:49,156 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0ce42bff9af84030a2a2fb64d3e4c48e 2023-11-19 23:53:49,157 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/ef4b5b73e2664c99b41b76402f85258e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf1/ef4b5b73e2664c99b41b76402f85258e 2023-11-19 23:53:49,158 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/0e87388c9e5d474ba6945dcdbe27e6c0 2023-11-19 23:53:49,159 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/760b2dbf36984af0864ac433c40afa53 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/cf2/760b2dbf36984af0864ac433c40afa53 2023-11-19 23:53:49,159 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/866684d3e7da4366a63f14630018cd35 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/cf2/866684d3e7da4366a63f14630018cd35 2023-11-19 23:53:49,160 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/63926b3f08874124b4dd6e937d54a044 2023-11-19 23:53:49,165 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/6a2cd21c82f54733be044f68ffc15eb9 2023-11-19 23:53:49,166 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/recovered.edits/113.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f/recovered.edits/113.seqid 2023-11-19 23:53:49,166 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/recovered.edits/112.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758/recovered.edits/112.seqid 2023-11-19 23:53:49,166 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:53:49,166 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a9c85c60733c72c1431e0d5b15931758 2023-11-19 23:53:49,167 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8642eb2f17a9462aa14b168d29a13dd8 2023-11-19 23:53:49,168 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/8964da69ac324f33a4579d0d8e2fb83c 2023-11-19 23:53:49,169 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/9c6c5b867a77484683c0e5f7bd1912a1 2023-11-19 23:53:49,170 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/c30edc8a799e4416931edcdb2b9201a3 2023-11-19 23:53:49,171 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/cd93de537f4c46948e31abe21427deeb 2023-11-19 23:53:49,172 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf1/e8497cb94e1e4cce955a86a981bd76cd 2023-11-19 23:53:49,174 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/29bd3d11a3944b7281eff4bf7f719e54 2023-11-19 23:53:49,175 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/2c0f90b1ccc54e7aa2a39476c84669bf 2023-11-19 23:53:49,176 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/51cfd63863cc4a47bcbd0f19f5732758 2023-11-19 23:53:49,177 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/7ffa31ca494f4264b6764ab0db097f41 2023-11-19 23:53:49,178 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/a104ba77f79a41dd92074a4731824c12 2023-11-19 23:53:49,179 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/c0720ce7632e453f9c26d498fb4ce6aa 2023-11-19 23:53:49,180 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/cea11d7c81084ff7b9cd1d6e55ffa9b5 2023-11-19 23:53:49,181 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d05a4d94ccd64f37a8fd1c4e0a39d3dc 2023-11-19 23:53:49,182 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/d7a46029b90b4f828c6b81c47d896197 2023-11-19 23:53:49,183 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/cf2/f34788f58c1b4dfab221c9fd4c1e73e6 2023-11-19 23:53:49,185 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/recovered.edits/106.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b/recovered.edits/106.seqid 2023-11-19 23:53:49,185 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7733ca898e07f4f307912b72a6278a2b 2023-11-19 23:53:49,185 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-11-19 23:53:49,187 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,188 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-11-19 23:53:49,190 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-11-19 23:53:49,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-11-19 23:53:49,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438029191"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:49,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438029191"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:49,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438029191"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:49,192 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:53:49,192 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7733ca898e07f4f307912b72a6278a2b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1700438025267.7733ca898e07f4f307912b72a6278a2b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a9c85c60733c72c1431e0d5b15931758, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1700438027419.a9c85c60733c72c1431e0d5b15931758.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 2c794c96882fdb1d29833ea1af058f2f, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1700438027419.2c794c96882fdb1d29833ea1af058f2f.', STARTKEY => '60', ENDKEY => ''}] 2023-11-19 23:53:49,193 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-11-19 23:53:49,193 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438029193"}]},"ts":"9223372036854775807"} 2023-11-19 23:53:49,194 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-11-19 23:53:49,196 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:53:49,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 51 msec 2023-11-19 23:53:49,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-11-19 23:53:49,249 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-11-19 23:53:49,263 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 408) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37696 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48232 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37668 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48210 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:40568 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37712 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48760 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48814 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37684 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37658 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:38404 [Waiting for operation #41] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:40642 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48746 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48774 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x329d2ba3-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:37228 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-197600505_17 at /127.0.0.1:48872 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=888 (was 884) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=368 (was 368), ProcessCount=171 (was 171), AvailableMemoryMB=15177 (was 15196) 2023-11-19 23:53:49,277 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=888, MaxFileDescriptor=60000, SystemLoadAverage=368, ProcessCount=171, AvailableMemoryMB=15176 2023-11-19 23:53:49,277 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:53:49,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:53:49,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:53:49,280 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:53:49,282 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:53:49,282 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:53:49,284 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,284 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb empty. 2023-11-19 23:53:49,284 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,284 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-11-19 23:53:49,295 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:49,296 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => a1bc841589c05bdbde058387f36c61bb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing a1bc841589c05bdbde058387f36c61bb, disabling compactions & flushes 2023-11-19 23:53:49,303 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. after waiting 0 ms 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,303 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,303 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for a1bc841589c05bdbde058387f36c61bb: 2023-11-19 23:53:49,304 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:53:49,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438029305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438029305"}]},"ts":"1700438029305"} 2023-11-19 23:53:49,306 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:53:49,307 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:53:49,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438029307"}]},"ts":"1700438029307"} 2023-11-19 23:53:49,308 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-11-19 23:53:49,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:49,318 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:49,319 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:49,319 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:49,319 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:53:49,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, ASSIGN}] 2023-11-19 23:53:49,320 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, ASSIGN 2023-11-19 23:53:49,321 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:53:49,471 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:49,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:49,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438029473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438029473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438029473"}]},"ts":"1700438029473"} 2023-11-19 23:53:49,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:49,625 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:49,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a1bc841589c05bdbde058387f36c61bb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:49,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:49,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:49,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,630 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,631 DEBUG [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:53:49,631 DEBUG [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:53:49,631 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a1bc841589c05bdbde058387f36c61bb columnFamilyName cf1 2023-11-19 23:53:49,632 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] regionserver.HStore(310): Store=a1bc841589c05bdbde058387f36c61bb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:49,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:49,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:53:49,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a1bc841589c05bdbde058387f36c61bb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68906262, jitterRate=0.02678331732749939}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-19 23:53:49,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a1bc841589c05bdbde058387f36c61bb: 2023-11-19 23:53:49,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb., pid=87, masterSystemTime=1700438029625 2023-11-19 23:53:49,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:49,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:49,641 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438029640"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438029640"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438029640"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438029640"}]},"ts":"1700438029640"} 2023-11-19 23:53:49,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=87, resume processing ppid=86 2023-11-19 23:53:49,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374 in 168 msec 2023-11-19 23:53:49,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=86, resume processing ppid=85 2023-11-19 23:53:49,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, ASSIGN in 325 msec 2023-11-19 23:53:49,646 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:53:49,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438029647"}]},"ts":"1700438029647"} 2023-11-19 23:53:49,648 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-11-19 23:53:49,651 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:53:49,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 371 msec 2023-11-19 23:53:49,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-19 23:53:49,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:53:49,950 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-11-19 23:53:49,951 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-11-19 23:53:49,952 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-11-19 23:53:49,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-11-19 23:53:49,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, REOPEN/MOVE}] 2023-11-19 23:53:49,967 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, REOPEN/MOVE 2023-11-19 23:53:49,968 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:49,968 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438029968"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438029968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438029968"}]},"ts":"1700438029968"} 2023-11-19 23:53:49,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:50,121 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:50,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a1bc841589c05bdbde058387f36c61bb, disabling compactions & flushes 2023-11-19 23:53:50,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. after waiting 0 ms 2023-11-19 23:53:50,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-11-19 23:53:50,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:53:50,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a1bc841589c05bdbde058387f36c61bb: 2023-11-19 23:53:50,126 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: a1bc841589c05bdbde058387f36c61bb to self. 2023-11-19 23:53:50,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=CLOSED 2023-11-19 23:53:50,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438030128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438030128"}]},"ts":"1700438030128"} 2023-11-19 23:53:50,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=91, resume processing ppid=90 2023-11-19 23:53:50,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374 in 160 msec 2023-11-19 23:53:50,132 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=true 2023-11-19 23:53:50,282 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:53:50,282 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:50,282 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438030282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438030282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438030282"}]},"ts":"1700438030282"} 2023-11-19 23:53:50,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:53:50,436 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:50,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a1bc841589c05bdbde058387f36c61bb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:53:50,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:50,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:50,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,440 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,441 DEBUG [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:53:50,441 DEBUG [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:53:50,442 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a1bc841589c05bdbde058387f36c61bb columnFamilyName cf1 2023-11-19 23:53:50,442 INFO [StoreOpener-a1bc841589c05bdbde058387f36c61bb-1] regionserver.HStore(310): Store=a1bc841589c05bdbde058387f36c61bb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:53:50,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:53:50,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a1bc841589c05bdbde058387f36c61bb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58881437, jitterRate=-0.12259821593761444}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-19 23:53:50,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a1bc841589c05bdbde058387f36c61bb: 2023-11-19 23:53:50,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb., pid=92, masterSystemTime=1700438030436 2023-11-19 23:53:50,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:53:50,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:53:50,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438030448"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438030448"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438030448"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438030448"}]},"ts":"1700438030448"} 2023-11-19 23:53:50,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=92, resume processing ppid=90 2023-11-19 23:53:50,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374 in 165 msec 2023-11-19 23:53:50,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=90, resume processing ppid=89 2023-11-19 23:53:50,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, REOPEN/MOVE in 485 msec 2023-11-19 23:53:50,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=89, resume processing ppid=88 2023-11-19 23:53:50,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-11-19 23:53:50,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, ASSIGN}] 2023-11-19 23:53:50,458 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, ASSIGN 2023-11-19 23:53:50,458 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-19 23:53:50,608 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:53:50,609 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:53:50,609 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:53:50,609 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:53:50,609 DEBUG [jenkins-hbase4:45383] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-11-19 23:53:50,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=707f65aebcc841dba894862f6eab2c45, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:50,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438030611"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1700438030611"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1700438030611"}]},"ts":"1700438030611"} 2023-11-19 23:53:50,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 707f65aebcc841dba894862f6eab2c45, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:53:50,767 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:53:50,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:53:50,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 707f65aebcc841dba894862f6eab2c45, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-11-19 23:53:50,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:53:50,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:53:50,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:53:50,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:53:50,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:53:50,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:50,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:50,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,857 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-11-19 23:53:51,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:51,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:52,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:53,975 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:53:53,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:54,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:55,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:56,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:57,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:58,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:53:59,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:00,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:01,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:02,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:03,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:04,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:05,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:06,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:07,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:08,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:09,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:10,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:11,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:12,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:13,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:14,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:15,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:16,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:17,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:18,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:19,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:20,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:21,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:22,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:23,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:24,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:25,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:26,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:27,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:28,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:29,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:30,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:31,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:32,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:33,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:34,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:35,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:36,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:37,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:38,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:39,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:40,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:41,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:42,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:43,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:44,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:45,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:46,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:47,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:48,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:49,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-19 23:54:49,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-19 23:54:50,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-19 23:54:50,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-19 23:54:50,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:50,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-19 23:54:51,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:51,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:52,011 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:52,012 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:52,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:52,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:52,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:52,015 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting a1bc841589c05bdbde058387f36c61bb, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1645) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:52,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1527): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 exec-time=6 msec 2023-11-19 23:54:52,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:52,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-11-19 23:54:52,283 INFO [StoreOpener-707f65aebcc841dba894862f6eab2c45-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:54:52,287 DEBUG [StoreOpener-707f65aebcc841dba894862f6eab2c45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:54:52,287 DEBUG [StoreOpener-707f65aebcc841dba894862f6eab2c45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1 2023-11-19 23:54:52,288 INFO [StoreOpener-707f65aebcc841dba894862f6eab2c45-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 707f65aebcc841dba894862f6eab2c45 columnFamilyName cf1 2023-11-19 23:54:52,289 INFO [StoreOpener-707f65aebcc841dba894862f6eab2c45-1] regionserver.HStore(310): Store=707f65aebcc841dba894862f6eab2c45/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:54:52,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 707f65aebcc841dba894862f6eab2c45; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73685604, jitterRate=0.09800106287002563}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-19 23:54:52,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 707f65aebcc841dba894862f6eab2c45: 2023-11-19 23:54:52,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45., pid=94, masterSystemTime=1700438030767 2023-11-19 23:54:52,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:52,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:52,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=707f65aebcc841dba894862f6eab2c45, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:52,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438092292"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1700438092292"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1700438092292"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1700438092292"}]},"ts":"1700438092292"} 2023-11-19 23:54:52,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=94, resume processing ppid=93 2023-11-19 23:54:52,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 707f65aebcc841dba894862f6eab2c45, server=jenkins-hbase4.apache.org,39613,1700438000411 in 1 mins, 1.68 sec 2023-11-19 23:54:52,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=93, resume processing ppid=88 2023-11-19 23:54:52,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, ASSIGN in 1 mins, 1.838 sec 2023-11-19 23:54:52,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.504 sec 2023-11-19 23:54:52,298 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1700438029787","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62511,"client":"172.31.14.131:38756","queuetimems":0,"class":"HMaster"} 2023-11-19 23:54:52,299 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(187): Unknown callId: 601, skipping over this response of 3 bytes 2023-11-19 23:54:54,263 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up 2023-11-19 23:54:54,264 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:54:54,266 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-19 23:54:54,271 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438094271"}]},"ts":"1700438094271"} 2023-11-19 23:54:54,272 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-11-19 23:54:54,277 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-11-19 23:54:54,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, UNASSIGN}] 2023-11-19 23:54:54,279 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, UNASSIGN 2023-11-19 23:54:54,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=707f65aebcc841dba894862f6eab2c45, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:54,280 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438094280"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1700438094280"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1700438094280"}]},"ts":"1700438094280"} 2023-11-19 23:54:54,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 707f65aebcc841dba894862f6eab2c45, server=jenkins-hbase4.apache.org,39613,1700438000411}] 2023-11-19 23:54:54,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, UNASSIGN 2023-11-19 23:54:54,285 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:54,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438094285"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438094285"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438094285"}]},"ts":"1700438094285"} 2023-11-19 23:54:54,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:54:54,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-19 23:54:54,434 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:54,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:54:54,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 707f65aebcc841dba894862f6eab2c45, disabling compactions & flushes 2023-11-19 23:54:54,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:54,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:54,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. after waiting 0 ms 2023-11-19 23:54:54,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:54,435 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:54:54,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45. 2023-11-19 23:54:54,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 707f65aebcc841dba894862f6eab2c45: 2023-11-19 23:54:54,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 707f65aebcc841dba894862f6eab2c45 2023-11-19 23:54:54,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=707f65aebcc841dba894862f6eab2c45, regionState=CLOSED 2023-11-19 23:54:54,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438094438"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1700438094438"}]},"ts":"1700438094438"} 2023-11-19 23:54:54,439 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:54,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:54:54,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a1bc841589c05bdbde058387f36c61bb, disabling compactions & flushes 2023-11-19 23:54:54,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:54:54,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:54:54,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. after waiting 0 ms 2023-11-19 23:54:54,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:54:54,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=99, resume processing ppid=98 2023-11-19 23:54:54,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 707f65aebcc841dba894862f6eab2c45, server=jenkins-hbase4.apache.org,39613,1700438000411 in 157 msec 2023-11-19 23:54:54,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45, UNASSIGN in 164 msec 2023-11-19 23:54:54,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-11-19 23:54:54,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:54:54,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb. 2023-11-19 23:54:54,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a1bc841589c05bdbde058387f36c61bb: 2023-11-19 23:54:54,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:54:54,447 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=a1bc841589c05bdbde058387f36c61bb, regionState=CLOSED 2023-11-19 23:54:54,447 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1700438094447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438094447"}]},"ts":"1700438094447"} 2023-11-19 23:54:54,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=100, resume processing ppid=97 2023-11-19 23:54:54,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure a1bc841589c05bdbde058387f36c61bb, server=jenkins-hbase4.apache.org,34057,1700438000374 in 162 msec 2023-11-19 23:54:54,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=97, resume processing ppid=96 2023-11-19 23:54:54,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a1bc841589c05bdbde058387f36c61bb, UNASSIGN in 171 msec 2023-11-19 23:54:54,451 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438094451"}]},"ts":"1700438094451"} 2023-11-19 23:54:54,452 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-11-19 23:54:54,455 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-11-19 23:54:54,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 188 msec 2023-11-19 23:54:54,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-19 23:54:54,572 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-11-19 23:54:54,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-11-19 23:54:54,581 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:54:54,582 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits] 2023-11-19 23:54:54,587 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits/7.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb/recovered.edits/7.seqid 2023-11-19 23:54:54,588 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a1bc841589c05bdbde058387f36c61bb 2023-11-19 23:54:54,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-11-19 23:54:54,590 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,592 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-11-19 23:54:54,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-11-19 23:54:54,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-11-19 23:54:54,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438094595"}]},"ts":"9223372036854775807"} 2023-11-19 23:54:54,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438094595"}]},"ts":"9223372036854775807"} 2023-11-19 23:54:54,596 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-11-19 23:54:54,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a1bc841589c05bdbde058387f36c61bb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280.a1bc841589c05bdbde058387f36c61bb.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 707f65aebcc841dba894862f6eab2c45, NAME => 'testRollbackForSplitTableRegionWithReplica,,1700438029280_0001.707f65aebcc841dba894862f6eab2c45.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-11-19 23:54:54,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-11-19 23:54:54,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438094597"}]},"ts":"9223372036854775807"} 2023-11-19 23:54:54,598 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-11-19 23:54:54,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:54,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 27 msec 2023-11-19 23:54:54,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-11-19 23:54:54,677 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-11-19 23:54:54,690 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=349 (was 414), OpenFileDescriptor=683 (was 888), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 368), ProcessCount=170 (was 171), AvailableMemoryMB=14881 (was 15176) 2023-11-19 23:54:54,702 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=350, OpenFileDescriptor=683, MaxFileDescriptor=60000, SystemLoadAverage=136, ProcessCount=170, AvailableMemoryMB=14881 2023-11-19 23:54:54,702 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:54:54,704 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:54:54,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:54:54,705 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:54,707 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:54:54,707 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:54:54,709 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:54,709 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 empty. 2023-11-19 23:54:54,710 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:54,710 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-11-19 23:54:54,718 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-11-19 23:54:54,719 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 190e51d652e09cd986dcc4f0501bb310, NAME => 'testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 190e51d652e09cd986dcc4f0501bb310, disabling compactions & flushes 2023-11-19 23:54:54,726 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. after waiting 0 ms 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:54,726 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:54,726 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:54,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:54:54,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1700438094728"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438094728"}]},"ts":"1700438094728"} 2023-11-19 23:54:54,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:54:54,730 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:54:54,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438094730"}]},"ts":"1700438094730"} 2023-11-19 23:54:54,731 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-11-19 23:54:54,736 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:54:54,736 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:54:54,736 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:54:54,736 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:54:54,737 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:54:54,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, ASSIGN}] 2023-11-19 23:54:54,738 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, ASSIGN 2023-11-19 23:54:54,738 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:54:54,888 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:54:54,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=190e51d652e09cd986dcc4f0501bb310, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:54,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1700438094889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438094889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438094889"}]},"ts":"1700438094889"} 2023-11-19 23:54:54,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 190e51d652e09cd986dcc4f0501bb310, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:54:55,043 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 190e51d652e09cd986dcc4f0501bb310, NAME => 'testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:54:55,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:54:55,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:54:55,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,049 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,050 DEBUG [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1 2023-11-19 23:54:55,050 DEBUG [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1 2023-11-19 23:54:55,051 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 190e51d652e09cd986dcc4f0501bb310 columnFamilyName cf1 2023-11-19 23:54:55,051 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] regionserver.HStore(310): Store=190e51d652e09cd986dcc4f0501bb310/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:54:55,051 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,052 DEBUG [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2 2023-11-19 23:54:55,052 DEBUG [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2 2023-11-19 23:54:55,053 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 190e51d652e09cd986dcc4f0501bb310 columnFamilyName cf2 2023-11-19 23:54:55,053 INFO [StoreOpener-190e51d652e09cd986dcc4f0501bb310-1] regionserver.HStore(310): Store=190e51d652e09cd986dcc4f0501bb310/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:54:55,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:54:55,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:55,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:54:55,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 190e51d652e09cd986dcc4f0501bb310; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66034472, jitterRate=-0.01600968837738037}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:54:55,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310., pid=104, masterSystemTime=1700438095043 2023-11-19 23:54:55,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=190e51d652e09cd986dcc4f0501bb310, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1700438095061"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438095061"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438095061"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438095061"}]},"ts":"1700438095061"} 2023-11-19 23:54:55,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=104, resume processing ppid=103 2023-11-19 23:54:55,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 190e51d652e09cd986dcc4f0501bb310, server=jenkins-hbase4.apache.org,36283,1700438000319 in 171 msec 2023-11-19 23:54:55,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=103, resume processing ppid=102 2023-11-19 23:54:55,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, ASSIGN in 327 msec 2023-11-19 23:54:55,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:54:55,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438095066"}]},"ts":"1700438095066"} 2023-11-19 23:54:55,067 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-11-19 23:54:55,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:54:55,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 364 msec 2023-11-19 23:54:55,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:55,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:55,216 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:55,216 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,216 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:55,216 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:55,217 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,217 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,218 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,218 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,218 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,218 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,219 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,219 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,221 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,221 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:55,222 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,222 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:55,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:55,223 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:54:55,224 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,226 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,226 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,226 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,226 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,226 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,235 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/e2c506aebbc64b86a2d93883c55f963e 2023-11-19 23:54:55,251 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/1d82ad67c7b54c33bc0761d73abbd49b 2023-11-19 23:54:55,255 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/e2c506aebbc64b86a2d93883c55f963e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/e2c506aebbc64b86a2d93883c55f963e 2023-11-19 23:54:55,259 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/e2c506aebbc64b86a2d93883c55f963e, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:54:55,259 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/1d82ad67c7b54c33bc0761d73abbd49b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/1d82ad67c7b54c33bc0761d73abbd49b 2023-11-19 23:54:55,263 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/1d82ad67c7b54c33bc0761d73abbd49b, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:54:55,264 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 41ms, sequenceid=6, compaction requested=false 2023-11-19 23:54:55,264 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-11-19 23:54:55,264 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,264 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,264 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:55,264 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:55,265 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,265 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,265 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,267 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,267 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,267 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,267 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:55,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4970d6e3[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:55,269 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:55,269 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:55,271 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,271 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:55,271 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:55,273 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,273 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,273 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,274 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:55,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:55,277 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:55,277 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:55,277 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:55,277 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:55,278 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,279 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,280 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,280 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,281 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,282 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,283 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,283 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:55,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:55,290 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,290 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,291 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:55,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,291 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:55,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,292 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,392 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:55,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:55,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:55,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:55,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:55,405 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:55,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:55,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:55,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,407 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,407 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,407 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:55,409 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:55,409 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,409 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:55,410 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,411 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,421 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/29fc4d6e642f4a19af09cecac3b1fd98 2023-11-19 23:54:55,432 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/a381a53efdea43b7af7e16d3b099cccd 2023-11-19 23:54:55,436 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/29fc4d6e642f4a19af09cecac3b1fd98 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98 2023-11-19 23:54:55,439 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:54:55,440 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/a381a53efdea43b7af7e16d3b099cccd as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd 2023-11-19 23:54:55,443 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:54:55,444 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 34ms, sequenceid=19, compaction requested=false 2023-11-19 23:54:55,444 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,444 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,444 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:55,444 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:55,444 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,444 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,444 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,446 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,446 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,447 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,447 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:55,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33aa2cbc[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:55,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:55,449 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:55,451 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,452 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:55,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:55,453 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,453 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,453 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,453 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,454 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:55,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:55,457 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:55,457 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:55,457 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:55,457 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:55,457 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,459 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,459 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,460 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,461 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,462 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,466 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,466 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,467 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:55,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:55,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,467 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:55,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:55,468 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,568 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:55,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:55,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:55,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:55,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:55,581 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:55,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:55,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:55,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,585 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:55,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:55,585 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,585 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:55,585 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:55,586 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,587 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,587 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,587 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,588 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,594 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/48c7cb37978d4d328b36030941b5eaf2 2023-11-19 23:54:55,610 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/d8af897eb4ff4cdb909363f5e61f644c 2023-11-19 23:54:55,615 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/48c7cb37978d4d328b36030941b5eaf2 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/48c7cb37978d4d328b36030941b5eaf2 2023-11-19 23:54:55,619 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/48c7cb37978d4d328b36030941b5eaf2, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:54:55,620 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/d8af897eb4ff4cdb909363f5e61f644c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d8af897eb4ff4cdb909363f5e61f644c 2023-11-19 23:54:55,633 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d8af897eb4ff4cdb909363f5e61f644c, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:54:55,634 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 49ms, sequenceid=32, compaction requested=true 2023-11-19 23:54:55,635 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,635 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,635 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:55,635 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:55,635 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,635 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,635 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,638 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,638 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:55,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f256b4a[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:55,641 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:55,641 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,642 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,642 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,642 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,643 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,643 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,645 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,645 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:55,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:55,653 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:55,653 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:55,653 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:55,653 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:55,653 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:55,658 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,658 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,658 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,658 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,658 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,659 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,659 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,659 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,660 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,662 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,662 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,663 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:55,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:55,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:55,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,671 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,672 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,772 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:55,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:55,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:55,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:55,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:55,786 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:55,786 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,786 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:55,786 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:55,786 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,786 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,787 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,788 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,788 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,789 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,790 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,790 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:55,790 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:55,790 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,790 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:55,790 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:55,792 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,794 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,794 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,794 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,794 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,794 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,803 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/6cc3ba3b51204670946f8853f2a3c419 2023-11-19 23:54:55,814 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/10b5fdb8b27547d0a1eacf9528971591 2023-11-19 23:54:55,818 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/6cc3ba3b51204670946f8853f2a3c419 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/6cc3ba3b51204670946f8853f2a3c419 2023-11-19 23:54:55,821 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/6cc3ba3b51204670946f8853f2a3c419, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:54:55,822 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/10b5fdb8b27547d0a1eacf9528971591 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/10b5fdb8b27547d0a1eacf9528971591 2023-11-19 23:54:55,825 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/10b5fdb8b27547d0a1eacf9528971591, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:54:55,826 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 36ms, sequenceid=45, compaction requested=true 2023-11-19 23:54:55,826 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,826 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,826 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:55,826 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:55,826 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,826 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,826 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,828 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,828 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,828 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,828 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:55,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e4a104[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:55,830 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:55,830 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,831 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,831 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,831 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,832 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:55,832 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:55,834 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,834 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,834 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,835 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,836 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:55,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:55,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:55,839 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:55,839 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:55,839 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:55,839 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:55,839 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:55,840 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,840 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:55,841 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,840 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,840 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:55,841 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,842 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:55,842 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:55,844 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:55,844 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,844 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,848 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:55,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:55,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:55,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:55,847 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:55,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,848 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:55,849 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:55,849 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,849 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,948 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:55,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:55,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:55,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:55,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:55,962 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:55,962 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,962 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:55,962 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:55,962 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,963 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,965 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,965 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,965 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:55,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:55,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:55,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:55,967 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:55,968 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:55,968 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,968 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:55,968 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:55,969 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,969 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,970 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,970 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,970 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:55,970 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:55,976 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/de310ddd637d44b0b45a475c9fae20fd 2023-11-19 23:54:55,987 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/d0819f19fc8c4500ba334f1dc34e10a9 2023-11-19 23:54:55,990 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/de310ddd637d44b0b45a475c9fae20fd as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/de310ddd637d44b0b45a475c9fae20fd 2023-11-19 23:54:55,994 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/de310ddd637d44b0b45a475c9fae20fd, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:54:55,995 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/d0819f19fc8c4500ba334f1dc34e10a9 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d0819f19fc8c4500ba334f1dc34e10a9 2023-11-19 23:54:55,998 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d0819f19fc8c4500ba334f1dc34e10a9, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:54:55,999 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 31ms, sequenceid=58, compaction requested=true 2023-11-19 23:54:55,999 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:55,999 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:56,001 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,001 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,001 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,001 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:56,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:56,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f6cb835[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:56,003 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:56,003 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,006 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,006 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:56,006 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,008 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,009 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,009 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,009 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,009 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:56,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:56,013 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:56,013 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:56,013 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:56,013 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:56,013 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,015 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,016 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,016 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,018 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,018 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,018 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:56,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:56,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,027 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:56,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:56,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,028 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,128 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:56,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:56,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:56,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:56,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-11-19 23:54:56,141 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-11-19 23:54:56,141 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:56,141 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-11-19 23:54:56,141 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-11-19 23:54:56,141 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,141 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,143 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,143 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,144 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:56,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:56,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:56,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:56,145 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-11-19 23:54:56,145 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. started... 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:56,145 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:56,145 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-11-19 23:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:56,146 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:56,147 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,147 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,148 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,148 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:56,148 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,148 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:56,154 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/ed7353784bf841159c195377e936b721 2023-11-19 23:54:56,164 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/06d6758cce1b48d8b419faf2825bc4a3 2023-11-19 23:54:56,167 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/ed7353784bf841159c195377e936b721 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/ed7353784bf841159c195377e936b721 2023-11-19 23:54:56,170 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/ed7353784bf841159c195377e936b721, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:54:56,171 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/06d6758cce1b48d8b419faf2825bc4a3 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/06d6758cce1b48d8b419faf2825bc4a3 2023-11-19 23:54:56,174 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/06d6758cce1b48d8b419faf2825bc4a3, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:54:56,175 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 30ms, sequenceid=71, compaction requested=true 2023-11-19 23:54:56,175 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:56,175 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-11-19 23:54:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-11-19 23:54:56,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,177 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,177 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-11-19 23:54:56,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d41a2a8[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:56,179 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-11-19 23:54:56,179 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:56,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,182 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:56,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-11-19 23:54:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:56,184 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,184 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,184 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,184 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,184 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,184 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,185 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,185 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-11-19 23:54:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,185 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,185 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-11-19 23:54:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:56,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-11-19 23:54:56,188 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-11-19 23:54:56,188 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:56,188 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:56,188 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-11-19 23:54:56,188 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,194 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,194 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,195 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:56,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-11-19 23:54:56,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,196 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,197 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,197 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,199 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-11-19 23:54:56,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,200 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:56,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:56,200 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-11-19 23:54:56,201 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-11-19 23:54:56,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:56,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:56,300 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:56,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:56,307 INFO [Listener at localhost/42385] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,308 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.StoreUtils(129): cannot split hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98 because midkey is the same as first or last row 2023-11-19 23:54:56,309 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.StoreUtils(129): cannot split hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd because midkey is the same as first or last row 2023-11-19 23:54:56,309 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HRegion(2745): Flushing 190e51d652e09cd986dcc4f0501bb310 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:54:56,317 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/3efdd55601f74893990faa9c35e5a8fb 2023-11-19 23:54:56,330 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/cdb1a9f81abf43539b354ebdd3b413b5 2023-11-19 23:54:56,333 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf1/3efdd55601f74893990faa9c35e5a8fb as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/3efdd55601f74893990faa9c35e5a8fb 2023-11-19 23:54:56,336 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/3efdd55601f74893990faa9c35e5a8fb, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:54:56,337 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/.tmp/cf2/cdb1a9f81abf43539b354ebdd3b413b5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/cdb1a9f81abf43539b354ebdd3b413b5 2023-11-19 23:54:56,340 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/cdb1a9f81abf43539b354ebdd3b413b5, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:54:56,341 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 190e51d652e09cd986dcc4f0501bb310 in 32ms, sequenceid=82, compaction requested=true 2023-11-19 23:54:56,341 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.HRegion(2446): Flush status journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:56,341 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.StoreUtils(129): cannot split hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98 because midkey is the same as first or last row 2023-11-19 23:54:56,341 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36283] regionserver.StoreUtils(129): cannot split hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd because midkey is the same as first or last row 2023-11-19 23:54:56,341 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,341 DEBUG [Listener at localhost/42385] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-11-19 23:54:56,341 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:54:56,343 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-11-19 23:54:56,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-11-19 23:54:56,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-19 23:54:56,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438096346"}]},"ts":"1700438096346"} 2023-11-19 23:54:56,347 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-11-19 23:54:56,353 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-11-19 23:54:56,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, UNASSIGN}] 2023-11-19 23:54:56,355 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, UNASSIGN 2023-11-19 23:54:56,356 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=190e51d652e09cd986dcc4f0501bb310, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,356 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1700438096356"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438096356"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438096356"}]},"ts":"1700438096356"} 2023-11-19 23:54:56,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 190e51d652e09cd986dcc4f0501bb310, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:54:56,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-19 23:54:56,508 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:56,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 190e51d652e09cd986dcc4f0501bb310, disabling compactions & flushes 2023-11-19 23:54:56,509 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. after waiting 0 ms 2023-11-19 23:54:56,509 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:54:56,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:54:56,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310. 2023-11-19 23:54:56,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 190e51d652e09cd986dcc4f0501bb310: 2023-11-19 23:54:56,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:56,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=190e51d652e09cd986dcc4f0501bb310, regionState=CLOSED 2023-11-19 23:54:56,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1700438096515"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438096515"}]},"ts":"1700438096515"} 2023-11-19 23:54:56,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=107, resume processing ppid=106 2023-11-19 23:54:56,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 190e51d652e09cd986dcc4f0501bb310, server=jenkins-hbase4.apache.org,36283,1700438000319 in 159 msec 2023-11-19 23:54:56,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=106, resume processing ppid=105 2023-11-19 23:54:56,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=190e51d652e09cd986dcc4f0501bb310, UNASSIGN in 163 msec 2023-11-19 23:54:56,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438096519"}]},"ts":"1700438096519"} 2023-11-19 23:54:56,520 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-11-19 23:54:56,521 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-11-19 23:54:56,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 179 msec 2023-11-19 23:54:56,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-19 23:54:56,648 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-11-19 23:54:56,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-11-19 23:54:56,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,651 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,651 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-11-19 23:54:56,654 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:56,656 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/recovered.edits] 2023-11-19 23:54:56,659 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/29fc4d6e642f4a19af09cecac3b1fd98 2023-11-19 23:54:56,660 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/3efdd55601f74893990faa9c35e5a8fb to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/3efdd55601f74893990faa9c35e5a8fb 2023-11-19 23:54:56,661 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/48c7cb37978d4d328b36030941b5eaf2 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/48c7cb37978d4d328b36030941b5eaf2 2023-11-19 23:54:56,662 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/6cc3ba3b51204670946f8853f2a3c419 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/6cc3ba3b51204670946f8853f2a3c419 2023-11-19 23:54:56,663 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/de310ddd637d44b0b45a475c9fae20fd to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/de310ddd637d44b0b45a475c9fae20fd 2023-11-19 23:54:56,664 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/e2c506aebbc64b86a2d93883c55f963e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/e2c506aebbc64b86a2d93883c55f963e 2023-11-19 23:54:56,665 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/ed7353784bf841159c195377e936b721 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf1/ed7353784bf841159c195377e936b721 2023-11-19 23:54:56,668 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/06d6758cce1b48d8b419faf2825bc4a3 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/06d6758cce1b48d8b419faf2825bc4a3 2023-11-19 23:54:56,669 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/10b5fdb8b27547d0a1eacf9528971591 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/10b5fdb8b27547d0a1eacf9528971591 2023-11-19 23:54:56,670 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/1d82ad67c7b54c33bc0761d73abbd49b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/1d82ad67c7b54c33bc0761d73abbd49b 2023-11-19 23:54:56,671 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/a381a53efdea43b7af7e16d3b099cccd 2023-11-19 23:54:56,672 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/cdb1a9f81abf43539b354ebdd3b413b5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/cdb1a9f81abf43539b354ebdd3b413b5 2023-11-19 23:54:56,673 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d0819f19fc8c4500ba334f1dc34e10a9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d0819f19fc8c4500ba334f1dc34e10a9 2023-11-19 23:54:56,674 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d8af897eb4ff4cdb909363f5e61f644c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/cf2/d8af897eb4ff4cdb909363f5e61f644c 2023-11-19 23:54:56,676 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310/recovered.edits/85.seqid 2023-11-19 23:54:56,677 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testInvalidSplitKey/190e51d652e09cd986dcc4f0501bb310 2023-11-19 23:54:56,677 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-11-19 23:54:56,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,680 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-11-19 23:54:56,683 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-11-19 23:54:56,684 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,684 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-11-19 23:54:56,684 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438096684"}]},"ts":"9223372036854775807"} 2023-11-19 23:54:56,685 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-19 23:54:56,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 190e51d652e09cd986dcc4f0501bb310, NAME => 'testInvalidSplitKey,,1700438094705.190e51d652e09cd986dcc4f0501bb310.', STARTKEY => '', ENDKEY => ''}] 2023-11-19 23:54:56,685 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-11-19 23:54:56,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438096685"}]},"ts":"9223372036854775807"} 2023-11-19 23:54:56,686 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-11-19 23:54:56,688 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:56,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 39 msec 2023-11-19 23:54:56,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-11-19 23:54:56,752 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-11-19 23:54:56,766 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=372 (was 350) - Thread LEAK? -, OpenFileDescriptor=770 (was 683) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=125 (was 136), ProcessCount=170 (was 170), AvailableMemoryMB=14875 (was 14881) 2023-11-19 23:54:56,778 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=372, OpenFileDescriptor=770, MaxFileDescriptor=60000, SystemLoadAverage=125, ProcessCount=170, AvailableMemoryMB=14875 2023-11-19 23:54:56,778 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:54:56,780 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:54:56,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:54:56,781 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:54:56,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:54:56,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:54:56,784 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:54:56,784 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 empty. 2023-11-19 23:54:56,784 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:54:56,785 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-11-19 23:54:56,793 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-11-19 23:54:56,794 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3e0179df131389dccfe85994dc503627, NAME => 'testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 3e0179df131389dccfe85994dc503627, disabling compactions & flushes 2023-11-19 23:54:56,800 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. after waiting 0 ms 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:56,800 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:56,800 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:56,802 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:54:56,802 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438096802"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438096802"}]},"ts":"1700438096802"} 2023-11-19 23:54:56,803 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:54:56,804 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:54:56,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438096804"}]},"ts":"1700438096804"} 2023-11-19 23:54:56,805 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-11-19 23:54:56,814 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:54:56,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:54:56,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:54:56,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:54:56,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:54:56,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, ASSIGN}] 2023-11-19 23:54:56,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, ASSIGN 2023-11-19 23:54:56,817 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:54:56,967 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:54:56,968 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3e0179df131389dccfe85994dc503627, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:56,968 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438096968"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438096968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438096968"}]},"ts":"1700438096968"} 2023-11-19 23:54:56,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:54:57,121 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3e0179df131389dccfe85994dc503627, NAME => 'testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:54:57,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:54:57,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:54:57,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,125 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,127 DEBUG [StoreOpener-3e0179df131389dccfe85994dc503627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1 2023-11-19 23:54:57,127 DEBUG [StoreOpener-3e0179df131389dccfe85994dc503627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1 2023-11-19 23:54:57,127 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e0179df131389dccfe85994dc503627 columnFamilyName cf1 2023-11-19 23:54:57,127 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] regionserver.HStore(310): Store=3e0179df131389dccfe85994dc503627/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:54:57,128 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,129 DEBUG [StoreOpener-3e0179df131389dccfe85994dc503627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2 2023-11-19 23:54:57,129 DEBUG [StoreOpener-3e0179df131389dccfe85994dc503627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2 2023-11-19 23:54:57,129 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e0179df131389dccfe85994dc503627 columnFamilyName cf2 2023-11-19 23:54:57,129 INFO [StoreOpener-3e0179df131389dccfe85994dc503627-1] regionserver.HStore(310): Store=3e0179df131389dccfe85994dc503627/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:54:57,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:54:57,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:57,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:54:57,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3e0179df131389dccfe85994dc503627; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64747050, jitterRate=-0.03519377112388611}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:54:57,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:57,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627., pid=111, masterSystemTime=1700438097121 2023-11-19 23:54:57,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3e0179df131389dccfe85994dc503627, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438097137"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438097137"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438097137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438097137"}]},"ts":"1700438097137"} 2023-11-19 23:54:57,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=111, resume processing ppid=110 2023-11-19 23:54:57,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 in 168 msec 2023-11-19 23:54:57,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=110, resume processing ppid=109 2023-11-19 23:54:57,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, ASSIGN in 324 msec 2023-11-19 23:54:57,141 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:54:57,141 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438097141"}]},"ts":"1700438097141"} 2023-11-19 23:54:57,142 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-11-19 23:54:57,144 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:54:57,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 364 msec 2023-11-19 23:54:57,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:57,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:57,289 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,289 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,289 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:57,289 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:57,289 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,289 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,291 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,291 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,292 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,293 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,293 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:57,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,293 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:57,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,295 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:54:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,297 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,298 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,298 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,298 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,298 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,299 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,304 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/3d783815564a4e24963f6d60386c3f7a 2023-11-19 23:54:57,314 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/32bc4795a52c4283a46c1ef5ddf10e22 2023-11-19 23:54:57,317 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/3d783815564a4e24963f6d60386c3f7a as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a 2023-11-19 23:54:57,321 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:54:57,321 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/32bc4795a52c4283a46c1ef5ddf10e22 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 2023-11-19 23:54:57,324 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:54:57,325 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 30ms, sequenceid=6, compaction requested=false 2023-11-19 23:54:57,325 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,325 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:57,325 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,327 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,327 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,327 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,327 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:57,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@796ae371[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:57,329 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:57,329 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:57,332 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,332 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:57,332 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,334 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,334 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,334 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,334 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,334 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,334 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,335 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:57,337 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:57,338 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:57,338 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:57,338 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,338 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,340 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,340 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,341 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,342 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,350 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:57,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,351 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,351 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,351 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,351 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,451 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:57,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:57,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:57,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:57,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:57,463 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:57,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:57,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,464 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,464 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,464 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,464 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,465 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,465 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:57,467 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,467 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,467 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:57,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,475 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,478 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/d6614f37ba2e40e8afea310b4609699e 2023-11-19 23:54:57,488 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/5881cffa6cb345b4b512678e4c4fb803 2023-11-19 23:54:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/d6614f37ba2e40e8afea310b4609699e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e 2023-11-19 23:54:57,495 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:54:57,495 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/5881cffa6cb345b4b512678e4c4fb803 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 2023-11-19 23:54:57,498 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:54:57,499 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 32ms, sequenceid=19, compaction requested=false 2023-11-19 23:54:57,499 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:57,499 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:57,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:57,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,499 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,501 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,501 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,501 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,501 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c2381d9[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:57,503 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:57,503 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,504 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,504 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:57,506 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:57,511 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:57,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:57,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:57,511 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,511 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,513 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,514 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,515 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,516 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,517 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,517 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,519 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,520 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:57,520 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,521 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,621 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:57,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:57,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:57,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:57,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:57,633 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,633 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,633 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:57,633 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:57,633 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,633 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,635 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,636 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,636 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:57,637 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:57,637 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,637 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:57,637 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,640 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,645 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/0ddbd61f41a646b381e97297b6d3848e 2023-11-19 23:54:57,655 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/898654e3964c4775b7a4338b84f43d3f 2023-11-19 23:54:57,659 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/0ddbd61f41a646b381e97297b6d3848e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e 2023-11-19 23:54:57,662 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:54:57,663 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/898654e3964c4775b7a4338b84f43d3f as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f 2023-11-19 23:54:57,666 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:54:57,667 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 30ms, sequenceid=32, compaction requested=true 2023-11-19 23:54:57,667 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:57,667 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,667 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:57,667 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:57,667 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,667 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,667 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,671 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:57,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76242778[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:57,674 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:57,674 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,675 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,676 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,677 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,677 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:57,684 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:57,684 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:57,684 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:57,684 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,684 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:57,685 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,685 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,685 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,685 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,685 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,686 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,687 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,688 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,689 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,691 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,692 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,693 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,693 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,693 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,792 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:57,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:57,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:57,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:57,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:57,805 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,805 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,805 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:57,805 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:57,805 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,805 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,806 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,806 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,806 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,806 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,807 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,807 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:57,809 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,809 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:57,809 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,817 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/cd3b720d19cc448783005d6cdbee02e5 2023-11-19 23:54:57,827 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/dfec8996f821498194f2e523553aa854 2023-11-19 23:54:57,831 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/cd3b720d19cc448783005d6cdbee02e5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 2023-11-19 23:54:57,835 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:54:57,835 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/dfec8996f821498194f2e523553aa854 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 2023-11-19 23:54:57,839 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:54:57,840 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 31ms, sequenceid=45, compaction requested=true 2023-11-19 23:54:57,840 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:57,840 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:57,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:57,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,840 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,842 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,842 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,842 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,842 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:57,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56509ff6[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:57,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:57,845 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:57,847 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:57,847 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:57,847 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,849 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:57,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:57,855 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:57,855 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:57,856 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:57,856 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,856 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,858 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:57,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,859 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,860 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,861 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,861 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,863 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:57,863 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:57,863 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,871 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,872 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,871 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:57,872 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:57,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,872 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:57,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:57,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:57,872 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,873 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,873 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:57,873 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,873 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,972 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:57,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:57,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:57,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:57,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:57,986 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:57,986 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,986 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:57,986 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:57,986 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,986 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,988 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,988 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:57,988 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:57,988 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:57,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:57,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,990 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,990 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:57,990 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:57,990 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:57,992 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,994 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:57,994 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:57,999 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b 2023-11-19 23:54:58,009 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/976a672aeccd4382b082a495d5ea7da8 2023-11-19 23:54:58,013 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b 2023-11-19 23:54:58,017 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:54:58,018 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/976a672aeccd4382b082a495d5ea7da8 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 2023-11-19 23:54:58,022 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:54:58,022 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 32ms, sequenceid=58, compaction requested=true 2023-11-19 23:54:58,023 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:58,023 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:58,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:58,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:58,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:58,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:58,025 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,025 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,025 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,025 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:58,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@780f8f26[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:58,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:58,027 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,029 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,029 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:58,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,031 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,032 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:58,035 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:58,035 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:58,035 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:58,035 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:58,035 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,038 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,039 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,039 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,040 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,042 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,042 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,043 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,046 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:58,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:58,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,046 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,047 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,147 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:58,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:58,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:54:58,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-11-19 23:54:58,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-11-19 23:54:58,159 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:58,159 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:58,159 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-11-19 23:54:58,159 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-11-19 23:54:58,160 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,160 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,161 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,161 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,161 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,161 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,161 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,162 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:54:58,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:58,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:54:58,163 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:54:58,163 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:58,164 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. started... 2023-11-19 23:54:58,164 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:54:58,168 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,169 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,169 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,169 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,169 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:58,169 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:58,172 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/6d4a13d41e30444a8c6409fc58209c48 2023-11-19 23:54:58,181 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/4bab0a98e3b341d79dec77ce5241c16d 2023-11-19 23:54:58,185 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/6d4a13d41e30444a8c6409fc58209c48 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 2023-11-19 23:54:58,188 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:54:58,188 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/4bab0a98e3b341d79dec77ce5241c16d as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d 2023-11-19 23:54:58,191 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:54:58,192 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 28ms, sequenceid=71, compaction requested=true 2023-11-19 23:54:58,192 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:58,192 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:58,192 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:54:58,192 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:54:58,192 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,192 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-11-19 23:54:58,192 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-11-19 23:54:58,194 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,194 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,194 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,194 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:54:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-11-19 23:54:58,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38e2e4b7[Count = 0] remaining members to acquire global barrier 2023-11-19 23:54:58,196 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-11-19 23:54:58,196 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,198 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,198 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:54:58,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-11-19 23:54:58,198 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:54:58,200 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,201 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,201 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,202 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,202 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-11-19 23:54:58,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:54:58,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-11-19 23:54:58,205 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-11-19 23:54:58,205 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:54:58,205 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:54:58,205 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-11-19 23:54:58,205 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:54:58,207 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,208 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,208 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:54:58,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-11-19 23:54:58,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,210 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,213 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,213 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,213 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-11-19 23:54:58,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:54:58,214 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:54:58,215 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,215 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:54:58,215 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,215 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-11-19 23:54:58,315 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:54:58,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:54:58,574 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-11-19 23:54:58,575 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:58,576 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,578 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:54:58,578 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:54:58,578 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:58,826 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,826 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:54:58,826 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:54:58,826 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:54:58,827 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-11-19 23:54:58,827 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:54:58,827 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:54:58,827 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:54:58,827 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:54:58,827 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:54:58,829 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-11-19 23:54:58,829 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:54:58,829 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:58,830 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:54:58,831 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:58,832 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:58,833 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:58,834 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:54:58,835 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,835 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:54:58,835 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,835 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,835 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,835 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:54:58,836 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:54:58,836 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:54:58,836 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438128836 2023-11-19 23:54:58,836 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:54:58,838 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,840 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:54:58,840 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:54:58,840 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:54:58,840 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438158840 2023-11-19 23:54:58,840 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438218840 2023-11-19 23:54:58,840 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:54:58,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:54:58,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:54:58,842 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:58,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:54:58,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:54:59,091 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:54:59,091 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:54:59,092 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:54:59,093 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-11-19 23:54:59,093 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:54:59,094 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:54:59,094 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:54:59,094 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:54:59,094 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,094 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,095 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:54:59,096 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,097 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:54:59,098 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,099 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:54:59,099 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,099 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,099 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,099 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:54:59,099 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:54:59,100 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:54:59,100 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438129100 2023-11-19 23:54:59,100 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:54:59,102 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,104 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:54:59,104 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:54:59,104 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:54:59,104 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438159104 2023-11-19 23:54:59,104 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438219104 2023-11-19 23:54:59,104 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:54:59,104 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:54:59,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN}] 2023-11-19 23:54:59,106 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:54:59,106 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:54:59,106 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:59,354 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:54:59,355 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:54:59,355 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:54:59,356 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:54:59,357 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-11-19 23:54:59,357 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:54:59,358 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:54:59,358 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:54:59,358 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:54:59,359 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,359 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:54:59,359 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,359 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,360 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,360 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:54:59,360 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,360 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,361 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,361 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:54:59,361 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:54:59,361 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,362 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,362 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,362 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:54:59,362 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,362 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,363 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,363 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:54:59,363 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,363 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,363 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,364 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,364 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:59,364 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:54:59,365 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,366 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:54:59,366 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,366 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,366 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,366 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 9 msec 2023-11-19 23:54:59,366 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:54:59,367 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:54:59,367 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438129367 2023-11-19 23:54:59,369 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:54:59,369 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,371 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:54:59,371 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:54:59,371 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:54:59,371 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438159371 2023-11-19 23:54:59,371 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438219371 2023-11-19 23:54:59,371 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:54:59,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN}] 2023-11-19 23:54:59,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:54:59,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:54:59,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3e0179df131389dccfe85994dc503627, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:59,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438099374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438099374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438099374"}]},"ts":"1700438099374"} 2023-11-19 23:54:59,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:54:59,526 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:59,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3e0179df131389dccfe85994dc503627, disabling compactions & flushes 2023-11-19 23:54:59,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:59,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:59,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. after waiting 0 ms 2023-11-19 23:54:59,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:59,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3e0179df131389dccfe85994dc503627 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:54:59,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/043c65cc97f54898aca6df051dc15139 2023-11-19 23:54:59,550 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/37194fcbd6354e6096ce3a0cedb9cd2c 2023-11-19 23:54:59,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf1/043c65cc97f54898aca6df051dc15139 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 2023-11-19 23:54:59,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:54:59,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/.tmp/cf2/37194fcbd6354e6096ce3a0cedb9cd2c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c 2023-11-19 23:54:59,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:54:59,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3e0179df131389dccfe85994dc503627 in 35ms, sequenceid=82, compaction requested=true 2023-11-19 23:54:59,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:54:59,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:54:59,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. 2023-11-19 23:54:59,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3e0179df131389dccfe85994dc503627: 2023-11-19 23:54:59,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3e0179df131389dccfe85994dc503627, regionState=CLOSED 2023-11-19 23:54:59,570 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438099569"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438099569"}]},"ts":"1700438099569"} 2023-11-19 23:54:59,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=114, resume processing ppid=113 2023-11-19 23:54:59,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 in 196 msec 2023-11-19 23:54:59,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=113, resume processing ppid=112 2023-11-19 23:54:59,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN in 201 msec 2023-11-19 23:54:59,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:54:59,575 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:54:59,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:59,621 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,621 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:54:59,621 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:54:59,622 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:54:59,622 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:54:59,623 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:54:59,624 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,625 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:54:59,626 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:54:59,627 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,628 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:54:59,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,629 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:54:59,629 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:54:59,630 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:54:59,630 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:54:59,631 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:54:59,631 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438129631 2023-11-19 23:54:59,631 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:54:59,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,635 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:54:59,635 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:54:59,635 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:54:59,635 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438159635 2023-11-19 23:54:59,635 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438219635 2023-11-19 23:54:59,635 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:54:59,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:54:59,649 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3e0179df131389dccfe85994dc503627, threads=14 2023-11-19 23:54:59,649 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,651 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,651 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,651 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,651 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,652 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,656 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,725 ERROR [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid 92965f72-b1d2-4a0b-81fa-7e4974137a5a) service to localhost/127.0.0.1:45125 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:54:59,832 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,833 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,847 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,851 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,852 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,852 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,852 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,853 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,864 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,866 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,866 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,867 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,872 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,878 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:54:59,878 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3e0179df131389dccfe85994dc503627 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:54:59,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:54:59,906 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:54:59,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:54:59,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:54:59,973 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:00,136 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:00,136 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:00,137 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:00,138 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-11-19 23:55:00,138 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:00,138 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:00,138 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,139 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:00,140 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,141 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,142 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:00,143 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,143 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,143 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,143 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,144 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,144 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:55:00,144 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:00,145 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:00,145 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438130145 2023-11-19 23:55:00,145 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:00,146 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,149 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:00,149 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:00,149 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:00,149 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438160149 2023-11-19 23:55:00,149 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438220149 2023-11-19 23:55:00,149 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:55:00,156 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3e0179df131389dccfe85994dc503627, threads=14 2023-11-19 23:55:00,156 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,156 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,157 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,157 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,157 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,158 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,159 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,159 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,226 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,237 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,245 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,246 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,247 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,252 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,253 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,256 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,258 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,258 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,259 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,259 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,261 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,263 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 for region: 3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,263 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3e0179df131389dccfe85994dc503627 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:55:00,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:00,294 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:00,295 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:00,295 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:00,295 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:00,295 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:00,399 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,399 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:00,399 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:00,399 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:00,400 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:00,400 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:00,400 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:00,400 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:00,400 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:00,400 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:00,401 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-11-19 23:55:00,401 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:00,401 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:00,402 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:00,402 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:00,402 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:00,402 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:00,402 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,403 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,404 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,405 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:00,406 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:00,406 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,407 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,407 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,407 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,407 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,407 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:55:00,407 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:00,408 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:00,408 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438130408 2023-11-19 23:55:00,408 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:00,410 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,412 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:00,412 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:00,412 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:00,412 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438160412 2023-11-19 23:55:00,412 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438220412 2023-11-19 23:55:00,412 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:55:00,414 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:00,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:00,416 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:00,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:00,662 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,662 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:00,663 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:00,663 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:00,664 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:00,665 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-11-19 23:55:00,665 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:00,665 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:00,665 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:00,665 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:00,666 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,667 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,668 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,669 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:00,670 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,670 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,670 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,670 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,670 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:55:00,671 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:00,671 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:00,671 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438130671 2023-11-19 23:55:00,673 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,673 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:00,675 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:00,675 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:00,675 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:00,676 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438160675 2023-11-19 23:55:00,676 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438220676 2023-11-19 23:55:00,676 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 2 msec 2023-11-19 23:55:00,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:00,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438100678"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1700438100678"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1700438100678"}]},"ts":"1700438100678"} 2023-11-19 23:55:00,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438100678"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438100678"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438100678"}]},"ts":"1700438100678"} 2023-11-19 23:55:00,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438100678"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438100678"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438100678"}]},"ts":"1700438100678"} 2023-11-19 23:55:00,680 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:00,680 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:00,680 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:00,926 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:00,926 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:00,927 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:00,927 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:00,928 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-11-19 23:55:00,928 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:00,928 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,929 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:00,930 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:00,931 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:00,932 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,933 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:00,934 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,934 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,934 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,934 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:00,934 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:55:00,935 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:00,936 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:00,936 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438130936 2023-11-19 23:55:00,936 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:00,937 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,939 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:00,940 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:00,940 INFO [Listener at localhost/42385] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319, table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627 2023-11-19 23:55:00,940 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ee7975a1668ee73616a1cc351dd045e1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:00,940 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=b9c6ae87beb3ec27a5f9359b946d508e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:00,940 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:00,940 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438160940 2023-11-19 23:55:00,940 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438220940 2023-11-19 23:55:00,940 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:55:00,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1700438100941"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1700438100941"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1700438100941"}]},"ts":"1700438100941"} 2023-11-19 23:55:00,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438100941"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438100941"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438100941"}]},"ts":"1700438100941"} 2023-11-19 23:55:00,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438100941"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438100941"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438100941"}]},"ts":"1700438100941"} 2023-11-19 23:55:00,943 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:00,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:00,944 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:00,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:00,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:01,190 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:01,191 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:01,191 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:01,192 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:01,193 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-11-19 23:55:01,193 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:01,193 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:01,193 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:01,193 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,194 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,195 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,196 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,197 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:01,198 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,198 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,198 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:01,198 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:01,198 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 6 msec 2023-11-19 23:55:01,199 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:01,200 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:01,200 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438131200 2023-11-19 23:55:01,200 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:01,201 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,204 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:01,204 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:01,204 INFO [Listener at localhost/42385] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319, table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627 2023-11-19 23:55:01,204 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ee7975a1668ee73616a1cc351dd045e1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:01,204 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=b9c6ae87beb3ec27a5f9359b946d508e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:01,204 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:01,204 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438161204 2023-11-19 23:55:01,205 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438221205 2023-11-19 23:55:01,205 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 4 msec 2023-11-19 23:55:01,205 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:01,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN}] 2023-11-19 23:55:01,206 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:01,207 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:01,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:01,455 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:01,455 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:01,456 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:01,458 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-11-19 23:55:01,458 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:01,458 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,459 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:01,460 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:01,461 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,462 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:01,462 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,462 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:01,462 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,462 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,463 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:01,464 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:01,464 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,464 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:01,464 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,464 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:01,464 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,464 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,465 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,465 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:01,465 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:01,465 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 8 msec 2023-11-19 23:55:01,465 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:01,467 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:01,474 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438131474 2023-11-19 23:55:01,476 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,476 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:01,478 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:01,478 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:01,479 INFO [Listener at localhost/42385] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319, table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627 2023-11-19 23:55:01,479 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ee7975a1668ee73616a1cc351dd045e1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:01,479 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=b9c6ae87beb3ec27a5f9359b946d508e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-11-19 23:55:01,479 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:01,479 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438161479 2023-11-19 23:55:01,479 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438221479 2023-11-19 23:55:01,479 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:55:01,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN}] 2023-11-19 23:55:01,480 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:01,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN 2023-11-19 23:55:01,482 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:55:01,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN 2023-11-19 23:55:01,483 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:55:01,633 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:55:01,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=ee7975a1668ee73616a1cc351dd045e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,635 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438101634"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438101634"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438101634"}]},"ts":"1700438101634"} 2023-11-19 23:55:01,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:01,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b9c6ae87beb3ec27a5f9359b946d508e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438101638"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438101638"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438101638"}]},"ts":"1700438101638"} 2023-11-19 23:55:01,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:01,787 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b9c6ae87beb3ec27a5f9359b946d508e, NAME => 'testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.', STARTKEY => '41', ENDKEY => ''} 2023-11-19 23:55:01,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:01,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:01,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,791 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,792 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1 2023-11-19 23:55:01,792 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1 2023-11-19 23:55:01,793 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9c6ae87beb3ec27a5f9359b946d508e columnFamilyName cf1 2023-11-19 23:55:01,803 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-top 2023-11-19 23:55:01,806 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-top 2023-11-19 23:55:01,809 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-top 2023-11-19 23:55:01,812 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-top 2023-11-19 23:55:01,815 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-top 2023-11-19 23:55:01,818 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-top 2023-11-19 23:55:01,821 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-top 2023-11-19 23:55:01,821 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(310): Store=b9c6ae87beb3ec27a5f9359b946d508e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:01,821 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,822 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2 2023-11-19 23:55:01,822 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2 2023-11-19 23:55:01,822 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9c6ae87beb3ec27a5f9359b946d508e columnFamilyName cf2 2023-11-19 23:55:01,833 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-top 2023-11-19 23:55:01,836 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-top 2023-11-19 23:55:01,839 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-top 2023-11-19 23:55:01,842 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-top 2023-11-19 23:55:01,845 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-top 2023-11-19 23:55:01,848 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-top 2023-11-19 23:55:01,851 DEBUG [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-top 2023-11-19 23:55:01,851 INFO [StoreOpener-b9c6ae87beb3ec27a5f9359b946d508e-1] regionserver.HStore(310): Store=b9c6ae87beb3ec27a5f9359b946d508e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:01,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:01,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b9c6ae87beb3ec27a5f9359b946d508e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66479906, jitterRate=-0.009372204542160034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:01,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b9c6ae87beb3ec27a5f9359b946d508e: 2023-11-19 23:55:01,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e., pid=118, masterSystemTime=1700438101787 2023-11-19 23:55:01,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:01,858 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:01,858 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:55:01,858 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): b9c6ae87beb3ec27a5f9359b946d508e/cf1 is initiating minor compaction (all files) 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,858 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): b9c6ae87beb3ec27a5f9359b946d508e/cf2 is initiating minor compaction (all files) 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b9c6ae87beb3ec27a5f9359b946d508e/cf1 in testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b9c6ae87beb3ec27a5f9359b946d508e/cf2 in testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/.tmp, totalSize=34.9 K 2023-11-19 23:55:01,858 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/.tmp, totalSize=34.9 K 2023-11-19 23:55:01,859 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438097283 2023-11-19 23:55:01,859 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438097283 2023-11-19 23:55:01,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:01,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,859 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438097452 2023-11-19 23:55:01,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee7975a1668ee73616a1cc351dd045e1, NAME => 'testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.', STARTKEY => '', ENDKEY => '41'} 2023-11-19 23:55:01,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b9c6ae87beb3ec27a5f9359b946d508e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,859 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438097452 2023-11-19 23:55:01,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:01,859 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438097622 2023-11-19 23:55:01,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438101859"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438101859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438101859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438101859"}]},"ts":"1700438101859"} 2023-11-19 23:55:01,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:01,860 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438097622 2023-11-19 23:55:01,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,860 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438097794 2023-11-19 23:55:01,860 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438097794 2023-11-19 23:55:01,861 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438097974 2023-11-19 23:55:01,861 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438098148 2023-11-19 23:55:01,861 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438097974 2023-11-19 23:55:01,861 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438098317 2023-11-19 23:55:01,862 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438098148 2023-11-19 23:55:01,862 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,862 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438098317 2023-11-19 23:55:01,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=118, resume processing ppid=116 2023-11-19 23:55:01,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319 in 222 msec 2023-11-19 23:55:01,863 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1 2023-11-19 23:55:01,863 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1 2023-11-19 23:55:01,864 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee7975a1668ee73616a1cc351dd045e1 columnFamilyName cf1 2023-11-19 23:55:01,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN in 383 msec 2023-11-19 23:55:01,881 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): b9c6ae87beb3ec27a5f9359b946d508e#cf1#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:01,888 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-bottom 2023-11-19 23:55:01,889 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): b9c6ae87beb3ec27a5f9359b946d508e#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:01,899 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-bottom 2023-11-19 23:55:01,913 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-bottom 2023-11-19 23:55:01,921 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-bottom 2023-11-19 23:55:01,924 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-bottom 2023-11-19 23:55:01,934 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-bottom 2023-11-19 23:55:01,938 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-bottom 2023-11-19 23:55:01,939 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(310): Store=ee7975a1668ee73616a1cc351dd045e1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:01,939 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,940 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2 2023-11-19 23:55:01,940 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2 2023-11-19 23:55:01,940 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee7975a1668ee73616a1cc351dd045e1 columnFamilyName cf2 2023-11-19 23:55:01,941 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/.tmp/cf2/f64830c181ef49e1be8f45e584697b88 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/f64830c181ef49e1be8f45e584697b88 2023-11-19 23:55:01,943 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/.tmp/cf1/0966ef43c0d74afd818309ad22e08cea as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0966ef43c0d74afd818309ad22e08cea 2023-11-19 23:55:01,946 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b9c6ae87beb3ec27a5f9359b946d508e/cf2 of b9c6ae87beb3ec27a5f9359b946d508e into f64830c181ef49e1be8f45e584697b88(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:01,946 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b9c6ae87beb3ec27a5f9359b946d508e: 2023-11-19 23:55:01,946 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e., storeName=b9c6ae87beb3ec27a5f9359b946d508e/cf2, priority=9, startTime=1700438101858; duration=0sec 2023-11-19 23:55:01,946 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:01,948 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b9c6ae87beb3ec27a5f9359b946d508e/cf1 of b9c6ae87beb3ec27a5f9359b946d508e into 0966ef43c0d74afd818309ad22e08cea(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:01,948 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b9c6ae87beb3ec27a5f9359b946d508e: 2023-11-19 23:55:01,948 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e., storeName=b9c6ae87beb3ec27a5f9359b946d508e/cf1, priority=9, startTime=1700438101857; duration=0sec 2023-11-19 23:55:01,948 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:01,952 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-bottom 2023-11-19 23:55:01,955 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-bottom 2023-11-19 23:55:01,958 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-bottom 2023-11-19 23:55:01,960 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-bottom 2023-11-19 23:55:01,963 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-bottom 2023-11-19 23:55:01,966 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-bottom 2023-11-19 23:55:01,969 DEBUG [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-bottom 2023-11-19 23:55:01,969 INFO [StoreOpener-ee7975a1668ee73616a1cc351dd045e1-1] regionserver.HStore(310): Store=ee7975a1668ee73616a1cc351dd045e1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:01,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:01,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:01,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee7975a1668ee73616a1cc351dd045e1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62754498, jitterRate=-0.06488510966300964}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:01,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee7975a1668ee73616a1cc351dd045e1: 2023-11-19 23:55:01,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1., pid=117, masterSystemTime=1700438101787 2023-11-19 23:55:01,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:01,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:55:01,975 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:01,974 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,975 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): ee7975a1668ee73616a1cc351dd045e1/cf2 is initiating minor compaction (all files) 2023-11-19 23:55:01,975 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): ee7975a1668ee73616a1cc351dd045e1/cf1 is initiating minor compaction (all files) 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ee7975a1668ee73616a1cc351dd045e1/cf2 in testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ee7975a1668ee73616a1cc351dd045e1/cf1 in testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/.tmp, totalSize=34.9 K 2023-11-19 23:55:01,975 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/.tmp, totalSize=34.9 K 2023-11-19 23:55:01,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,976 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438097283 2023-11-19 23:55:01,976 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438097283 2023-11-19 23:55:01,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:01,976 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438097452 2023-11-19 23:55:01,976 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438097452 2023-11-19 23:55:01,976 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=ee7975a1668ee73616a1cc351dd045e1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:01,977 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438101976"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438101976"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438101976"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438101976"}]},"ts":"1700438101976"} 2023-11-19 23:55:01,977 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438097622 2023-11-19 23:55:01,977 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438097622 2023-11-19 23:55:01,977 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438097794 2023-11-19 23:55:01,977 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438097794 2023-11-19 23:55:01,978 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438097974 2023-11-19 23:55:01,978 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438097974 2023-11-19 23:55:01,978 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438098148 2023-11-19 23:55:01,978 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438098148 2023-11-19 23:55:01,979 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438098317 2023-11-19 23:55:01,979 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438098317 2023-11-19 23:55:01,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=117, resume processing ppid=115 2023-11-19 23:55:01,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319 in 342 msec 2023-11-19 23:55:01,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=115, resume processing ppid=112 2023-11-19 23:55:01,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN in 500 msec 2023-11-19 23:55:01,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-11-19 23:55:01,981 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1758): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:01,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:01,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1979): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1760) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1702) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1391) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1963) 2023-11-19 23:55:01,988 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): ee7975a1668ee73616a1cc351dd045e1#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:01,989 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): ee7975a1668ee73616a1cc351dd045e1#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:02,003 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/.tmp/cf2/dcf292a7d3c14449b0a93ddb22ac34ee as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dcf292a7d3c14449b0a93ddb22ac34ee 2023-11-19 23:55:02,004 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/.tmp/cf1/c9ce42a5fd7947b88f31106b091b886c as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/c9ce42a5fd7947b88f31106b091b886c 2023-11-19 23:55:02,007 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ee7975a1668ee73616a1cc351dd045e1/cf2 of ee7975a1668ee73616a1cc351dd045e1 into dcf292a7d3c14449b0a93ddb22ac34ee(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:02,007 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ee7975a1668ee73616a1cc351dd045e1: 2023-11-19 23:55:02,007 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1., storeName=ee7975a1668ee73616a1cc351dd045e1/cf2, priority=9, startTime=1700438101974; duration=0sec 2023-11-19 23:55:02,007 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:02,008 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ee7975a1668ee73616a1cc351dd045e1/cf1 of ee7975a1668ee73616a1cc351dd045e1 into c9ce42a5fd7947b88f31106b091b886c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:02,008 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ee7975a1668ee73616a1cc351dd045e1: 2023-11-19 23:55:02,008 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1., storeName=ee7975a1668ee73616a1cc351dd045e1/cf1, priority=9, startTime=1700438101974; duration=0sec 2023-11-19 23:55:02,008 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:02,230 ERROR [Listener at localhost/42385] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(561): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-11-19 23:55:02,230 INFO [Listener at localhost/42385] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-19 23:55:02,231 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 0 msec 2023-11-19 23:55:02,232 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-11-19 23:55:02,232 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-11-19 23:55:02,232 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-11-19 23:55:02,232 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-11-19 23:55:02,232 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-11-19 23:55:02,233 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=dae2bb8e46806b529da56f5c659c8d9b, daughterA=5b5dc1190d203ff79365b9903c40512a, daughterB=31dbf9d0416d92b720428ba12f6265d3 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=515c2e69ac3f877374dbf5e5cd3d1f8d, daughterA=0c4a4342de5afa19f4fa73194454b024, daughterB=0b50a5610148ac95f35e1e3c642779e7 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-11-19 23:55:02,234 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ad9b2a7a0288c9e75c03b4a758726fb9, daughterA=4c377b2f8ce4a84a38366317178f7b91, daughterB=380f57fc7e4f191d259216af1345c0d9 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ad9b2a7a0288c9e75c03b4a758726fb9, ASSIGN 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=5876f55847b223c4fd0e2c973b172fc6, daughterA=c06cfd0a72f03a2218b666309cabf009, daughterB=936870a1b2b873ee71f15effa88fe4b2 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:02,235 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7733ca898e07f4f307912b72a6278a2b, daughterA=a9c85c60733c72c1431e0d5b15931758, daughterB=2c794c96882fdb1d29833ea1af058f2f 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:02,236 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39613,1700438000411, table=testRollbackForSplitTableRegionWithReplica, region=707f65aebcc841dba894862f6eab2c45 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a1bc841589c05bdbde058387f36c61bb, daughterA=7547c24fd33f753fee629166458733a2, daughterB=05d130b529e4271d8d6c09d27f173d34 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-11-19 23:55:02,237 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(410): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(467): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,238 DEBUG [Listener at localhost/42385] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e held the lock before restarting, call acquireLock to restore it. 2023-11-19 23:55:02,239 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,239 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,239 INFO [Listener at localhost/42385] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e0179df131389dccfe85994dc503627, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 DEBUG [Listener at localhost/42385] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319 didn't hold the lock before restarting, skip acquiring lock. 2023-11-19 23:55:02,239 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 7 msec 2023-11-19 23:55:02,240 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-11-19 23:55:02,240 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,39613,1700438000411, table=hbase:meta, region=1588230740 2023-11-19 23:55:02,240 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1700438132240 2023-11-19 23:55:02,242 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1399): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,242 DEBUG [Listener at localhost/42385] assignment.AssignmentManager(1527): Joining cluster... 2023-11-19 23:55:02,245 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=329994419c4b206406658a465b604c17, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39613,1700438000411, regionLocation=jenkins-hbase4.apache.org,39613,1700438000411, openSeqNum=2 2023-11-19 23:55:02,245 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=3e0179df131389dccfe85994dc503627, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=2 2023-11-19 23:55:02,245 INFO [Listener at localhost/42385] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36283,1700438000319, table=testRecoveryAndDoubleExecution, region=3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,245 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=ee7975a1668ee73616a1cc351dd045e1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=86 2023-11-19 23:55:02,246 DEBUG [Listener at localhost/42385] assignment.RegionStateStore(147): Load hbase:meta entry region=b9c6ae87beb3ec27a5f9359b946d508e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36283,1700438000319, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319, openSeqNum=86 2023-11-19 23:55:02,246 INFO [Listener at localhost/42385] assignment.AssignmentManager(1539): Number of RegionServers=3 2023-11-19 23:55:02,246 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1700438162246 2023-11-19 23:55:02,246 INFO [Listener at localhost/42385] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1700438222246 2023-11-19 23:55:02,246 INFO [Listener at localhost/42385] assignment.AssignmentManager(1546): Joined the cluster in 3 msec 2023-11-19 23:55:02,246 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-11-19 23:55:02,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e0179df131389dccfe85994dc503627, daughterA=ee7975a1668ee73616a1cc351dd045e1, daughterB=b9c6ae87beb3ec27a5f9359b946d508e in 3.6710 sec 2023-11-19 23:55:02,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36283] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:48854 deadline: 1700438162497, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627. is not online on jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,607 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:55:02,613 INFO [Listener at localhost/42385] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-11-19 23:55:02,615 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:55:02,621 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:55:02,623 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-11-19 23:55:02,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-11-19 23:55:02,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-11-19 23:55:02,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438102626"}]},"ts":"1700438102626"} 2023-11-19 23:55:02,627 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-11-19 23:55:02,628 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-11-19 23:55:02,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, UNASSIGN}] 2023-11-19 23:55:02,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, UNASSIGN 2023-11-19 23:55:02,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b9c6ae87beb3ec27a5f9359b946d508e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438102630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438102630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438102630"}]},"ts":"1700438102630"} 2023-11-19 23:55:02,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:02,632 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, UNASSIGN 2023-11-19 23:55:02,633 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=ee7975a1668ee73616a1cc351dd045e1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438102633"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438102633"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438102633"}]},"ts":"1700438102633"} 2023-11-19 23:55:02,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:02,667 ERROR [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid c1e6d43a-1c05-42be-a588-44ff607a74aa) service to localhost/127.0.0.1:45125 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:55:02,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-11-19 23:55:02,782 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:02,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:02,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee7975a1668ee73616a1cc351dd045e1, disabling compactions & flushes 2023-11-19 23:55:02,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:02,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:02,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. after waiting 0 ms 2023-11-19 23:55:02,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:02,783 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-bottom] to archive 2023-11-19 23:55:02,784 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:02,785 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,786 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,788 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,789 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,790 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,791 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,792 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,794 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-bottom] to archive 2023-11-19 23:55:02,794 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:02,795 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,797 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,798 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,799 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,800 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,801 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,802 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:55:02,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:02,805 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1. 2023-11-19 23:55:02,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee7975a1668ee73616a1cc351dd045e1: 2023-11-19 23:55:02,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:02,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b9c6ae87beb3ec27a5f9359b946d508e, disabling compactions & flushes 2023-11-19 23:55:02,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:02,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:02,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. after waiting 0 ms 2023-11-19 23:55:02,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:02,807 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=ee7975a1668ee73616a1cc351dd045e1, regionState=CLOSED 2023-11-19 23:55:02,808 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438102807"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438102807"}]},"ts":"1700438102807"} 2023-11-19 23:55:02,809 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139-top] to archive 2023-11-19 23:55:02,809 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:02,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=123, resume processing ppid=120 2023-11-19 23:55:02,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure ee7975a1668ee73616a1cc351dd045e1, server=jenkins-hbase4.apache.org,36283,1700438000319 in 175 msec 2023-11-19 23:55:02,811 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ee7975a1668ee73616a1cc351dd045e1, UNASSIGN in 181 msec 2023-11-19 23:55:02,812 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,813 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,814 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,815 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,816 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,817 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,818 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c-top] to archive 2023-11-19 23:55:02,819 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:02,820 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,821 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,821 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,822 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,823 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,824 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,825 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:55:02,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:02,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e. 2023-11-19 23:55:02,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b9c6ae87beb3ec27a5f9359b946d508e: 2023-11-19 23:55:02,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,830 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b9c6ae87beb3ec27a5f9359b946d508e, regionState=CLOSED 2023-11-19 23:55:02,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1700438102830"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438102830"}]},"ts":"1700438102830"} 2023-11-19 23:55:02,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=122, resume processing ppid=121 2023-11-19 23:55:02,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure b9c6ae87beb3ec27a5f9359b946d508e, server=jenkins-hbase4.apache.org,36283,1700438000319 in 200 msec 2023-11-19 23:55:02,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=121, resume processing ppid=119 2023-11-19 23:55:02,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b9c6ae87beb3ec27a5f9359b946d508e, UNASSIGN in 203 msec 2023-11-19 23:55:02,834 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438102834"}]},"ts":"1700438102834"} 2023-11-19 23:55:02,835 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-11-19 23:55:02,836 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-11-19 23:55:02,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 214 msec 2023-11-19 23:55:02,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-11-19 23:55:02,927 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-11-19 23:55:02,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-11-19 23:55:02,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,931 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-11-19 23:55:02,935 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,935 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:02,936 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,937 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/recovered.edits] 2023-11-19 23:55:02,938 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/recovered.edits] 2023-11-19 23:55:02,938 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/recovered.edits] 2023-11-19 23:55:02,942 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/.regioninfo to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/.regioninfo 2023-11-19 23:55:02,942 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/c9ce42a5fd7947b88f31106b091b886c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf1/c9ce42a5fd7947b88f31106b091b886c 2023-11-19 23:55:02,942 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/043c65cc97f54898aca6df051dc15139 2023-11-19 23:55:02,944 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/0ddbd61f41a646b381e97297b6d3848e 2023-11-19 23:55:02,945 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dcf292a7d3c14449b0a93ddb22ac34ee to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/cf2/dcf292a7d3c14449b0a93ddb22ac34ee 2023-11-19 23:55:02,945 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b 2023-11-19 23:55:02,945 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,946 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/3d783815564a4e24963f6d60386c3f7a 2023-11-19 23:55:02,947 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,948 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/6d4a13d41e30444a8c6409fc58209c48 2023-11-19 23:55:02,948 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/.regioninfo to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/.regioninfo 2023-11-19 23:55:02,949 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,950 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/cd3b720d19cc448783005d6cdbee02e5 2023-11-19 23:55:02,950 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,951 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf1/d6614f37ba2e40e8afea310b4609699e 2023-11-19 23:55:02,952 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,952 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/043c65cc97f54898aca6df051dc15139.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,953 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/0ddbd61f41a646b381e97297b6d3848e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,953 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,954 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/32bc4795a52c4283a46c1ef5ddf10e22 2023-11-19 23:55:02,955 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/1d1b7a86cb2f4c50af5bf39ae82bf83b.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,955 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,956 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/37194fcbd6354e6096ce3a0cedb9cd2c 2023-11-19 23:55:02,957 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/3d783815564a4e24963f6d60386c3f7a.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,958 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/4bab0a98e3b341d79dec77ce5241c16d 2023-11-19 23:55:02,959 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/6d4a13d41e30444a8c6409fc58209c48.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,959 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,960 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/cd3b720d19cc448783005d6cdbee02e5.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,961 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf1/d6614f37ba2e40e8afea310b4609699e.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,963 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/32bc4795a52c4283a46c1ef5ddf10e22.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,965 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,966 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,967 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,968 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/5881cffa6cb345b4b512678e4c4fb803 2023-11-19 23:55:02,969 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,969 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/37194fcbd6354e6096ce3a0cedb9cd2c.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,974 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/898654e3964c4775b7a4338b84f43d3f 2023-11-19 23:55:02,974 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,975 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/4bab0a98e3b341d79dec77ce5241c16d.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,976 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/976a672aeccd4382b082a495d5ea7da8 2023-11-19 23:55:02,976 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/ee7975a1668ee73616a1cc351dd045e1/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,976 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/5881cffa6cb345b4b512678e4c4fb803.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,978 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/cf2/dfec8996f821498194f2e523553aa854 2023-11-19 23:55:02,978 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/898654e3964c4775b7a4338b84f43d3f.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,980 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/976a672aeccd4382b082a495d5ea7da8.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,982 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1/recovered.edits/90.seqid 2023-11-19 23:55:02,982 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627/recovered.edits/85.seqid 2023-11-19 23:55:02,982 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/b9c6ae87beb3ec27a5f9359b946d508e/cf2/dfec8996f821498194f2e523553aa854.3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,983 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/ee7975a1668ee73616a1cc351dd045e1 2023-11-19 23:55:02,983 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/3e0179df131389dccfe85994dc503627 2023-11-19 23:55:02,985 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0966ef43c0d74afd818309ad22e08cea to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf1/0966ef43c0d74afd818309ad22e08cea 2023-11-19 23:55:02,987 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/f64830c181ef49e1be8f45e584697b88 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/cf2/f64830c181ef49e1be8f45e584697b88 2023-11-19 23:55:02,990 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e/recovered.edits/90.seqid 2023-11-19 23:55:02,990 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testRecoveryAndDoubleExecution/b9c6ae87beb3ec27a5f9359b946d508e 2023-11-19 23:55:02,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-11-19 23:55:02,993 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,995 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-11-19 23:55:02,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-11-19 23:55:02,998 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:02,998 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-11-19 23:55:02,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438102998"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:02,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438102998"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:02,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438102998"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:03,000 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:55:03,000 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3e0179df131389dccfe85994dc503627, NAME => 'testRecoveryAndDoubleExecution,,1700438096780.3e0179df131389dccfe85994dc503627.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => ee7975a1668ee73616a1cc351dd045e1, NAME => 'testRecoveryAndDoubleExecution,,1700438098575.ee7975a1668ee73616a1cc351dd045e1.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b9c6ae87beb3ec27a5f9359b946d508e, NAME => 'testRecoveryAndDoubleExecution,41,1700438098575.b9c6ae87beb3ec27a5f9359b946d508e.', STARTKEY => '41', ENDKEY => ''}] 2023-11-19 23:55:03,000 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-11-19 23:55:03,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438103001"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:03,002 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-11-19 23:55:03,004 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-11-19 23:55:03,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 76 msec 2023-11-19 23:55:03,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-11-19 23:55:03,032 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-11-19 23:55:03,048 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=380 (was 372) - Thread LEAK? -, OpenFileDescriptor=869 (was 770) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=131 (was 125) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=14850 (was 14875) 2023-11-19 23:55:03,063 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=380, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=131, ProcessCount=170, AvailableMemoryMB=14850 2023-11-19 23:55:03,064 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:55:03,066 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:55:03,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:55:03,067 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:55:03,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:55:03,071 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,071 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf empty. 2023-11-19 23:55:03,071 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,071 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-11-19 23:55:03,081 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-11-19 23:55:03,082 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 758657921095a87a597d099024cf4cdf, NAME => 'testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 758657921095a87a597d099024cf4cdf, disabling compactions & flushes 2023-11-19 23:55:03,088 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. after waiting 0 ms 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,088 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,088 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:03,090 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:55:03,091 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438103091"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438103091"}]},"ts":"1700438103091"} 2023-11-19 23:55:03,092 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:55:03,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:55:03,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438103093"}]},"ts":"1700438103093"} 2023-11-19 23:55:03,094 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-11-19 23:55:03,098 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:55:03,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:55:03,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:55:03,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:55:03,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:55:03,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, ASSIGN}] 2023-11-19 23:55:03,100 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, ASSIGN 2023-11-19 23:55:03,101 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:55:03,251 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:55:03,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=758657921095a87a597d099024cf4cdf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,252 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438103252"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438103252"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438103252"}]},"ts":"1700438103252"} 2023-11-19 23:55:03,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 758657921095a87a597d099024cf4cdf, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:03,405 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 758657921095a87a597d099024cf4cdf, NAME => 'testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:55:03,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:03,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:03,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,409 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,410 DEBUG [StoreOpener-758657921095a87a597d099024cf4cdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1 2023-11-19 23:55:03,410 DEBUG [StoreOpener-758657921095a87a597d099024cf4cdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1 2023-11-19 23:55:03,410 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 758657921095a87a597d099024cf4cdf columnFamilyName cf1 2023-11-19 23:55:03,411 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] regionserver.HStore(310): Store=758657921095a87a597d099024cf4cdf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:03,411 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,412 DEBUG [StoreOpener-758657921095a87a597d099024cf4cdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2 2023-11-19 23:55:03,412 DEBUG [StoreOpener-758657921095a87a597d099024cf4cdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2 2023-11-19 23:55:03,412 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 758657921095a87a597d099024cf4cdf columnFamilyName cf2 2023-11-19 23:55:03,412 INFO [StoreOpener-758657921095a87a597d099024cf4cdf-1] regionserver.HStore(310): Store=758657921095a87a597d099024cf4cdf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:03,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:03,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:03,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:55:03,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 758657921095a87a597d099024cf4cdf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61673598, jitterRate=-0.08099177479743958}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:03,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:03,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf., pid=127, masterSystemTime=1700438103405 2023-11-19 23:55:03,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=758657921095a87a597d099024cf4cdf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,419 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438103419"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438103419"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438103419"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438103419"}]},"ts":"1700438103419"} 2023-11-19 23:55:03,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=127, resume processing ppid=126 2023-11-19 23:55:03,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 758657921095a87a597d099024cf4cdf, server=jenkins-hbase4.apache.org,34057,1700438000374 in 166 msec 2023-11-19 23:55:03,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=126, resume processing ppid=125 2023-11-19 23:55:03,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, ASSIGN in 322 msec 2023-11-19 23:55:03,423 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:55:03,424 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438103424"}]},"ts":"1700438103424"} 2023-11-19 23:55:03,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-11-19 23:55:03,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:55:03,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 361 msec 2023-11-19 23:55:03,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:03,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,575 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,575 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,575 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:03,575 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:03,576 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,576 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,584 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:03,585 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,585 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,585 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,586 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:55:03,588 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,596 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/bae65117e86641ae9324a04a1ec2e9b9 2023-11-19 23:55:03,606 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/cdd504e23a424284938baaf6e26c723b 2023-11-19 23:55:03,609 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/bae65117e86641ae9324a04a1ec2e9b9 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9 2023-11-19 23:55:03,612 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:55:03,612 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/cdd504e23a424284938baaf6e26c723b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b 2023-11-19 23:55:03,615 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:55:03,615 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 29ms, sequenceid=6, compaction requested=false 2023-11-19 23:55:03,615 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,616 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:03,616 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,616 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:03,616 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:03,616 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,616 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,616 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,619 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,619 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,619 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,619 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:03,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6faa0b34[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:03,621 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:03,621 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,622 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,623 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,623 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:03,625 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,625 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,625 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,633 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:03,635 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:03,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:03,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:03,635 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,635 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,637 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,638 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,639 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,639 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,644 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,644 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,645 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,649 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,649 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,649 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,649 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,650 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:03,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,650 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:03,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,651 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,751 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:03,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:03,762 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:03,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:03,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,763 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:03,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:03,763 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,764 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,765 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,765 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,765 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:03,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,767 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,767 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:03,767 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,767 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:03,770 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,775 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/7e150658e5024f3f9c25816bd3670929 2023-11-19 23:55:03,785 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/5c49990d6ede4a25ba62c83492438e0a 2023-11-19 23:55:03,788 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/7e150658e5024f3f9c25816bd3670929 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929 2023-11-19 23:55:03,791 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:55:03,792 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/5c49990d6ede4a25ba62c83492438e0a as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a 2023-11-19 23:55:03,795 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:55:03,796 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 29ms, sequenceid=19, compaction requested=false 2023-11-19 23:55:03,796 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:03,796 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,796 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:03,796 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:03,796 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,796 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,796 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,798 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,798 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,798 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,798 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dd79618[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:03,800 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:03,800 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,801 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,801 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,802 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,802 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:03,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:03,802 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:03,804 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,804 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,804 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,804 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,805 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:03,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:03,808 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:03,808 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:03,808 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:03,808 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,808 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:03,812 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,812 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,812 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,812 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,813 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,812 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,813 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,813 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,814 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,816 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,816 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,816 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:03,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,820 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:03,819 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,819 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,820 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:03,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,820 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,820 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,821 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,821 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,821 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,920 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:03,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:03,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:03,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:03,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,933 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,933 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,933 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:03,933 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:03,933 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,933 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,935 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,935 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:03,937 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:03,937 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,937 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,938 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,940 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,941 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,952 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/e31a25b1ee114456839f0f049c3e668f 2023-11-19 23:55:03,962 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/f558cbf59b0e4b34bb9a5c351618f8bc 2023-11-19 23:55:03,965 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/e31a25b1ee114456839f0f049c3e668f as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f 2023-11-19 23:55:03,968 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:55:03,969 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/f558cbf59b0e4b34bb9a5c351618f8bc as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc 2023-11-19 23:55:03,972 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:55:03,973 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 36ms, sequenceid=32, compaction requested=true 2023-11-19 23:55:03,973 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:03,973 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:03,973 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:03,973 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:03,973 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,973 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:03,973 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:03,975 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,975 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,976 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,976 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:03,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39a5583b[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:03,978 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:03,978 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,980 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:03,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,979 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,979 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:03,980 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:03,983 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,983 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:03,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:03,987 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:03,987 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:03,987 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:03,987 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,987 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:03,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,990 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,990 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,991 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:03,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,993 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:03,993 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,993 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:03,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:03,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,997 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:03,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,997 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:03,998 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,098 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:04,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:04,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:04,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:04,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,110 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,110 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,110 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:04,110 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:04,111 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,111 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,119 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,119 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,119 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:04,121 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,121 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,121 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,122 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:04,123 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,124 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,124 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,124 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,124 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,124 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,130 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/1983dfe3216d46b7876d3dce9782b341 2023-11-19 23:55:04,139 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/63983129e8d141ee9c75e9d5fc06d9ea 2023-11-19 23:55:04,142 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/1983dfe3216d46b7876d3dce9782b341 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341 2023-11-19 23:55:04,145 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:55:04,146 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/63983129e8d141ee9c75e9d5fc06d9ea as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea 2023-11-19 23:55:04,149 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:55:04,149 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 28ms, sequenceid=45, compaction requested=true 2023-11-19 23:55:04,149 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:04,149 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,149 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,152 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,152 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:04,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16cdc4e4[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:04,154 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:04,154 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,155 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,155 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,155 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,156 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,156 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:04,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:04,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:04,156 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:04,157 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,157 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,157 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,160 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,160 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,160 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:04,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:04,163 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:04,163 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:04,163 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:04,163 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,163 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,164 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,165 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,165 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,166 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,167 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,168 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,168 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,171 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,171 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,172 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,271 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:04,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:04,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:04,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:04,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,284 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:04,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:04,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,284 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,286 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,286 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,286 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,286 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,286 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,287 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,287 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,287 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:04,288 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,288 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,288 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,288 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,291 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,296 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe 2023-11-19 23:55:04,310 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/147580c53bc9446da786bb6449fcad57 2023-11-19 23:55:04,314 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe 2023-11-19 23:55:04,316 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:55:04,317 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/147580c53bc9446da786bb6449fcad57 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57 2023-11-19 23:55:04,320 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:55:04,321 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 33ms, sequenceid=58, compaction requested=true 2023-11-19 23:55:04,321 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:04,321 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,323 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,323 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:04,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29818ea2[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:04,325 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:04,325 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,326 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,326 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,327 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:04,326 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,327 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,331 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,332 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,332 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,335 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,335 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:04,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:04,336 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:04,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:04,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:04,336 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,336 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,338 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,339 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,339 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,340 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,343 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:04,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,348 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,348 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,349 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,350 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,449 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:04,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:04,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:04,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:04,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,463 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-11-19 23:55:04,463 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-11-19 23:55:04,464 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,464 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,465 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,465 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,465 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,465 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,466 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,466 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,468 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-11-19 23:55:04,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:04,469 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,469 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. started... 2023-11-19 23:55:04,469 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:04,470 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,470 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,471 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,471 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,471 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,471 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,479 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/dc06d12bdbed4a48a9e647a03ebbc034 2023-11-19 23:55:04,892 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/c6766443920742f090e593a23a6f084b 2023-11-19 23:55:04,897 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/dc06d12bdbed4a48a9e647a03ebbc034 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034 2023-11-19 23:55:04,900 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:55:04,901 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/c6766443920742f090e593a23a6f084b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b 2023-11-19 23:55:04,904 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:55:04,905 INFO [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 436ms, sequenceid=71, compaction requested=true 2023-11-19 23:55:04,905 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:04,905 DEBUG [rs(jenkins-hbase4.apache.org,34057,1700438000374)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:04,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:04,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:04,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-11-19 23:55:04,905 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-11-19 23:55:04,907 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,907 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,907 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,907 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-11-19 23:55:04,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2867badc[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:04,910 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-11-19 23:55:04,910 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,911 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-11-19 23:55:04,911 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:04,912 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,912 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,913 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,913 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,913 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,913 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,913 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,915 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-11-19 23:55:04,915 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,915 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-11-19 23:55:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,34057,1700438000374': 2023-11-19 23:55:04,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-11-19 23:55:04,918 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-11-19 23:55:04,918 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:04,918 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:04,918 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,918 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:04,919 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,919 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,919 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,920 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,920 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,919 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:04,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,921 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,921 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:04,923 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,923 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,923 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,929 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:04,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-11-19 23:55:04,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:04,929 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:04,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:04,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:04,930 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:05,030 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:05,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:05,037 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,038 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=758657921095a87a597d099024cf4cdf, daughterA=6f55933bcd938d42ee92cfc06520a6f5, daughterB=2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=758657921095a87a597d099024cf4cdf, daughterA=6f55933bcd938d42ee92cfc06520a6f5, daughterB=2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=758657921095a87a597d099024cf4cdf, daughterA=6f55933bcd938d42ee92cfc06520a6f5, daughterB=2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=758657921095a87a597d099024cf4cdf, daughterA=6f55933bcd938d42ee92cfc06520a6f5, daughterB=2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, UNASSIGN}] 2023-11-19 23:55:05,041 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, UNASSIGN 2023-11-19 23:55:05,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=758657921095a87a597d099024cf4cdf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,042 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438105042"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438105042"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438105042"}]},"ts":"1700438105042"} 2023-11-19 23:55:05,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 758657921095a87a597d099024cf4cdf, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:05,068 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:55:05,194 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 758657921095a87a597d099024cf4cdf, disabling compactions & flushes 2023-11-19 23:55:05,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:05,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:05,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. after waiting 0 ms 2023-11-19 23:55:05,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:05,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 758657921095a87a597d099024cf4cdf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:55:05,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/67624ea155d44c3780619638bcfdcf16 2023-11-19 23:55:05,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/9873c766eb824fdb8dddd539fc295576 2023-11-19 23:55:05,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf1/67624ea155d44c3780619638bcfdcf16 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16 2023-11-19 23:55:05,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:55:05,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.tmp/cf2/9873c766eb824fdb8dddd539fc295576 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576 2023-11-19 23:55:05,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:55:05,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 758657921095a87a597d099024cf4cdf in 28ms, sequenceid=82, compaction requested=true 2023-11-19 23:55:05,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:55:05,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:05,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. 2023-11-19 23:55:05,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 758657921095a87a597d099024cf4cdf: 2023-11-19 23:55:05,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,229 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=758657921095a87a597d099024cf4cdf, regionState=CLOSED 2023-11-19 23:55:05,229 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438105229"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438105229"}]},"ts":"1700438105229"} 2023-11-19 23:55:05,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=130, resume processing ppid=129 2023-11-19 23:55:05,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 758657921095a87a597d099024cf4cdf, server=jenkins-hbase4.apache.org,34057,1700438000374 in 187 msec 2023-11-19 23:55:05,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=129, resume processing ppid=128 2023-11-19 23:55:05,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=758657921095a87a597d099024cf4cdf, UNASSIGN in 191 msec 2023-11-19 23:55:05,241 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=758657921095a87a597d099024cf4cdf, threads=14 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,242 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,243 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,243 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,243 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,245 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b, top=false 2023-11-19 23:55:05,247 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929, top=false 2023-11-19 23:55:05,248 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576, top=false 2023-11-19 23:55:05,248 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe, top=false 2023-11-19 23:55:05,248 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f, top=false 2023-11-19 23:55:05,250 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a, top=false 2023-11-19 23:55:05,250 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16, top=false 2023-11-19 23:55:05,251 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc, top=false 2023-11-19 23:55:05,254 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341, top=false 2023-11-19 23:55:05,254 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57, top=false 2023-11-19 23:55:05,255 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034, top=false 2023-11-19 23:55:05,256 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea, top=false 2023-11-19 23:55:05,259 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b, top=false 2023-11-19 23:55:05,261 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9, top=false 2023-11-19 23:55:05,264 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,266 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,266 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,266 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,271 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,272 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,273 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,273 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,276 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,279 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,286 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,286 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,286 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,288 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,289 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,305 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,306 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,306 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,306 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,307 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,309 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,309 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,309 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,309 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/.splits/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341 for child: 6f55933bcd938d42ee92cfc06520a6f5, parent: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,310 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,312 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,313 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,313 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341 for region: 758657921095a87a597d099024cf4cdf 2023-11-19 23:55:05,313 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 758657921095a87a597d099024cf4cdf Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-11-19 23:55:05,336 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:05,338 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:05,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1700438105339"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1700438105339"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1700438105339"}]},"ts":"1700438105339"} 2023-11-19 23:55:05,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105339"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438105339"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438105339"}]},"ts":"1700438105339"} 2023-11-19 23:55:05,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105339"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438105339"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438105339"}]},"ts":"1700438105339"} 2023-11-19 23:55:05,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, ASSIGN}] 2023-11-19 23:55:05,342 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, ASSIGN 2023-11-19 23:55:05,343 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:55:05,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, ASSIGN 2023-11-19 23:55:05,344 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34057,1700438000374; forceNewPlan=false, retain=false 2023-11-19 23:55:05,493 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:55:05,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=2f62bcef5efb6b2b9f2b4f8143d46f0c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105495"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438105495"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438105495"}]},"ts":"1700438105495"} 2023-11-19 23:55:05,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 2f62bcef5efb6b2b9f2b4f8143d46f0c, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:05,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6f55933bcd938d42ee92cfc06520a6f5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105498"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438105498"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438105498"}]},"ts":"1700438105498"} 2023-11-19 23:55:05,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 6f55933bcd938d42ee92cfc06520a6f5, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:05,647 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:05,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6f55933bcd938d42ee92cfc06520a6f5, NAME => 'testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.', STARTKEY => '', ENDKEY => '71'} 2023-11-19 23:55:05,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:05,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:05,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,651 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,652 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1 2023-11-19 23:55:05,652 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1 2023-11-19 23:55:05,653 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f55933bcd938d42ee92cfc06520a6f5 columnFamilyName cf1 2023-11-19 23:55:05,657 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341 2023-11-19 23:55:05,660 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16 2023-11-19 23:55:05,662 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929 2023-11-19 23:55:05,664 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe 2023-11-19 23:55:05,666 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9 2023-11-19 23:55:05,668 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034 2023-11-19 23:55:05,671 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f 2023-11-19 23:55:05,671 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(310): Store=6f55933bcd938d42ee92cfc06520a6f5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:05,671 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,672 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2 2023-11-19 23:55:05,672 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2 2023-11-19 23:55:05,672 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f55933bcd938d42ee92cfc06520a6f5 columnFamilyName cf2 2023-11-19 23:55:05,676 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57 2023-11-19 23:55:05,679 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a 2023-11-19 23:55:05,681 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea 2023-11-19 23:55:05,683 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576 2023-11-19 23:55:05,686 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b 2023-11-19 23:55:05,688 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b 2023-11-19 23:55:05,690 DEBUG [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc 2023-11-19 23:55:05,691 INFO [StoreOpener-6f55933bcd938d42ee92cfc06520a6f5-1] regionserver.HStore(310): Store=6f55933bcd938d42ee92cfc06520a6f5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:05,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:05,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:05,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6f55933bcd938d42ee92cfc06520a6f5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59545927, jitterRate=-0.11269654333591461}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:05,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6f55933bcd938d42ee92cfc06520a6f5: 2023-11-19 23:55:05,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5., pid=134, masterSystemTime=1700438105647 2023-11-19 23:55:05,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:05,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1912): 6f55933bcd938d42ee92cfc06520a6f5/cf1 is initiating minor compaction (all files) 2023-11-19 23:55:05,697 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1912): 6f55933bcd938d42ee92cfc06520a6f5/cf2 is initiating minor compaction (all files) 2023-11-19 23:55:05,697 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6f55933bcd938d42ee92cfc06520a6f5/cf1 in testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:05,697 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6f55933bcd938d42ee92cfc06520a6f5/cf2 in testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:05,697 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/.tmp, totalSize=34.9 K 2023-11-19 23:55:05,697 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/.tmp, totalSize=34.9 K 2023-11-19 23:55:05,698 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438103570 2023-11-19 23:55:05,698 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438103570 2023-11-19 23:55:05,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:05,698 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438103752 2023-11-19 23:55:05,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:05,698 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438103752 2023-11-19 23:55:05,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:05,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2f62bcef5efb6b2b9f2b4f8143d46f0c, NAME => 'testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.', STARTKEY => '71', ENDKEY => ''} 2023-11-19 23:55:05,699 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6f55933bcd938d42ee92cfc06520a6f5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,699 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438103922 2023-11-19 23:55:05,699 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438103922 2023-11-19 23:55:05,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:05,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105698"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438105698"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438105698"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438105698"}]},"ts":"1700438105698"} 2023-11-19 23:55:05,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,699 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438104099 2023-11-19 23:55:05,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:05,699 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438104099 2023-11-19 23:55:05,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,699 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438104273 2023-11-19 23:55:05,700 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438104273 2023-11-19 23:55:05,700 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438104451 2023-11-19 23:55:05,700 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438104451 2023-11-19 23:55:05,700 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438105032 2023-11-19 23:55:05,701 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,701 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438105032 2023-11-19 23:55:05,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=134, resume processing ppid=131 2023-11-19 23:55:05,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 6f55933bcd938d42ee92cfc06520a6f5, server=jenkins-hbase4.apache.org,34057,1700438000374 in 202 msec 2023-11-19 23:55:05,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, ASSIGN in 361 msec 2023-11-19 23:55:05,704 DEBUG [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf1 2023-11-19 23:55:05,704 DEBUG [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf1 2023-11-19 23:55:05,705 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f62bcef5efb6b2b9f2b4f8143d46f0c columnFamilyName cf1 2023-11-19 23:55:05,705 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] regionserver.HStore(310): Store=2f62bcef5efb6b2b9f2b4f8143d46f0c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:05,705 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,707 DEBUG [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf2 2023-11-19 23:55:05,707 DEBUG [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf2 2023-11-19 23:55:05,707 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f62bcef5efb6b2b9f2b4f8143d46f0c columnFamilyName cf2 2023-11-19 23:55:05,708 INFO [StoreOpener-2f62bcef5efb6b2b9f2b4f8143d46f0c-1] regionserver.HStore(310): Store=2f62bcef5efb6b2b9f2b4f8143d46f0c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:05,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:05,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:05,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2f62bcef5efb6b2b9f2b4f8143d46f0c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58794957, jitterRate=-0.12388686835765839}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:05,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2f62bcef5efb6b2b9f2b4f8143d46f0c: 2023-11-19 23:55:05,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c., pid=133, masterSystemTime=1700438105647 2023-11-19 23:55:05,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:05,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:05,716 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=2f62bcef5efb6b2b9f2b4f8143d46f0c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,717 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105716"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438105716"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438105716"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438105716"}]},"ts":"1700438105716"} 2023-11-19 23:55:05,720 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6f55933bcd938d42ee92cfc06520a6f5#cf1#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:05,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=133, resume processing ppid=132 2023-11-19 23:55:05,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 2f62bcef5efb6b2b9f2b4f8143d46f0c, server=jenkins-hbase4.apache.org,34057,1700438000374 in 222 msec 2023-11-19 23:55:05,720 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] throttle.PressureAwareThroughputController(145): 6f55933bcd938d42ee92cfc06520a6f5#cf2#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:05,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=132, resume processing ppid=128 2023-11-19 23:55:05,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, ASSIGN in 378 msec 2023-11-19 23:55:05,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=758657921095a87a597d099024cf4cdf, daughterA=6f55933bcd938d42ee92cfc06520a6f5, daughterB=2f62bcef5efb6b2b9f2b4f8143d46f0c in 684 msec 2023-11-19 23:55:05,731 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/.tmp/cf1/7d7110a56c6b4d49bbfe1824e855d493 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/7d7110a56c6b4d49bbfe1824e855d493 2023-11-19 23:55:05,732 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/.tmp/cf2/27ee39b549724a20a62ef47878cf3a05 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/27ee39b549724a20a62ef47878cf3a05 2023-11-19 23:55:05,735 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f55933bcd938d42ee92cfc06520a6f5/cf2 of 6f55933bcd938d42ee92cfc06520a6f5 into 27ee39b549724a20a62ef47878cf3a05(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:05,735 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f55933bcd938d42ee92cfc06520a6f5/cf1 of 6f55933bcd938d42ee92cfc06520a6f5 into 7d7110a56c6b4d49bbfe1824e855d493(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:05,735 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6f55933bcd938d42ee92cfc06520a6f5: 2023-11-19 23:55:05,736 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6f55933bcd938d42ee92cfc06520a6f5: 2023-11-19 23:55:05,736 INFO [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5., storeName=6f55933bcd938d42ee92cfc06520a6f5/cf2, priority=9, startTime=1700438105697; duration=0sec 2023-11-19 23:55:05,736 INFO [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5., storeName=6f55933bcd938d42ee92cfc06520a6f5/cf1, priority=9, startTime=1700438105696; duration=0sec 2023-11-19 23:55:05,736 DEBUG [RS:1;jenkins-hbase4:34057-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:05,736 DEBUG [RS:1;jenkins-hbase4:34057-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:05,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34057] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:56758 deadline: 1700438165789, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf. is not online on jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,898 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-11-19 23:55:05,901 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:55:05,902 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-11-19 23:55:05,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-11-19 23:55:05,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-11-19 23:55:05,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438105906"}]},"ts":"1700438105906"} 2023-11-19 23:55:05,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-11-19 23:55:05,908 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-11-19 23:55:05,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, UNASSIGN}] 2023-11-19 23:55:05,910 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, UNASSIGN 2023-11-19 23:55:05,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=2f62bcef5efb6b2b9f2b4f8143d46f0c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,910 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] assignment.AssignmentManager(1312): Unable to acquire lock for regionNode state=CLOSING, location=jenkins-hbase4.apache.org,34057,1700438000374, table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now. 2023-11-19 23:55:05,910 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438105910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438105910"}]},"ts":"1700438105910"} 2023-11-19 23:55:05,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 2f62bcef5efb6b2b9f2b4f8143d46f0c, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:05,913 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, UNASSIGN 2023-11-19 23:55:05,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6f55933bcd938d42ee92cfc06520a6f5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:05,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438105914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438105914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438105914"}]},"ts":"1700438105914"} 2023-11-19 23:55:05,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 6f55933bcd938d42ee92cfc06520a6f5, server=jenkins-hbase4.apache.org,34057,1700438000374}] 2023-11-19 23:55:06,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-11-19 23:55:06,063 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:06,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2f62bcef5efb6b2b9f2b4f8143d46f0c, disabling compactions & flushes 2023-11-19 23:55:06,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:06,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:06,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. after waiting 0 ms 2023-11-19 23:55:06,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:06,067 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-11-19 23:55:06,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:06,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c. 2023-11-19 23:55:06,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2f62bcef5efb6b2b9f2b4f8143d46f0c: 2023-11-19 23:55:06,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:06,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:06,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6f55933bcd938d42ee92cfc06520a6f5, disabling compactions & flushes 2023-11-19 23:55:06,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:06,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:06,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. after waiting 0 ms 2023-11-19 23:55:06,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:06,069 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=2f62bcef5efb6b2b9f2b4f8143d46f0c, regionState=CLOSED 2023-11-19 23:55:06,069 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438106069"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438106069"}]},"ts":"1700438106069"} 2023-11-19 23:55:06,070 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16] to archive 2023-11-19 23:55:06,070 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:06,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=138, resume processing ppid=137 2023-11-19 23:55:06,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 2f62bcef5efb6b2b9f2b4f8143d46f0c, server=jenkins-hbase4.apache.org,34057,1700438000374 in 159 msec 2023-11-19 23:55:06,072 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-bae65117e86641ae9324a04a1ec2e9b9 2023-11-19 23:55:06,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2f62bcef5efb6b2b9f2b4f8143d46f0c, UNASSIGN in 163 msec 2023-11-19 23:55:06,073 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-7e150658e5024f3f9c25816bd3670929 2023-11-19 23:55:06,074 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-e31a25b1ee114456839f0f049c3e668f 2023-11-19 23:55:06,075 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-1983dfe3216d46b7876d3dce9782b341 2023-11-19 23:55:06,076 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9101d5a9a2db4fd79d2d4bd5ca36f5fe 2023-11-19 23:55:06,077 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-dc06d12bdbed4a48a9e647a03ebbc034 2023-11-19 23:55:06,078 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-67624ea155d44c3780619638bcfdcf16 2023-11-19 23:55:06,079 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576] to archive 2023-11-19 23:55:06,080 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:06,081 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-cdd504e23a424284938baaf6e26c723b 2023-11-19 23:55:06,082 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-5c49990d6ede4a25ba62c83492438e0a 2023-11-19 23:55:06,083 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-f558cbf59b0e4b34bb9a5c351618f8bc 2023-11-19 23:55:06,084 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-63983129e8d141ee9c75e9d5fc06d9ea 2023-11-19 23:55:06,085 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-147580c53bc9446da786bb6449fcad57 2023-11-19 23:55:06,086 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-c6766443920742f090e593a23a6f084b 2023-11-19 23:55:06,086 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/testSplitTableRegionEmptyDaughter=758657921095a87a597d099024cf4cdf-9873c766eb824fdb8dddd539fc295576 2023-11-19 23:55:06,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:55:06,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:06,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5. 2023-11-19 23:55:06,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6f55933bcd938d42ee92cfc06520a6f5: 2023-11-19 23:55:06,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:06,091 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6f55933bcd938d42ee92cfc06520a6f5, regionState=CLOSED 2023-11-19 23:55:06,091 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1700438106091"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438106091"}]},"ts":"1700438106091"} 2023-11-19 23:55:06,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=139, resume processing ppid=136 2023-11-19 23:55:06,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 6f55933bcd938d42ee92cfc06520a6f5, server=jenkins-hbase4.apache.org,34057,1700438000374 in 176 msec 2023-11-19 23:55:06,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=136, resume processing ppid=135 2023-11-19 23:55:06,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6f55933bcd938d42ee92cfc06520a6f5, UNASSIGN in 184 msec 2023-11-19 23:55:06,095 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438106095"}]},"ts":"1700438106095"} 2023-11-19 23:55:06,096 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-11-19 23:55:06,098 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-11-19 23:55:06,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-11-19 23:55:06,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-11-19 23:55:06,207 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-11-19 23:55:06,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,210 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,211 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-11-19 23:55:06,215 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:06,215 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:06,215 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:06,217 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/recovered.edits] 2023-11-19 23:55:06,217 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/recovered.edits] 2023-11-19 23:55:06,218 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/recovered.edits] 2023-11-19 23:55:06,221 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/7d7110a56c6b4d49bbfe1824e855d493 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf1/7d7110a56c6b4d49bbfe1824e855d493 2023-11-19 23:55:06,221 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/1983dfe3216d46b7876d3dce9782b341 2023-11-19 23:55:06,223 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/67624ea155d44c3780619638bcfdcf16 2023-11-19 23:55:06,224 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/recovered.edits/88.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c/recovered.edits/88.seqid 2023-11-19 23:55:06,224 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/27ee39b549724a20a62ef47878cf3a05 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/cf2/27ee39b549724a20a62ef47878cf3a05 2023-11-19 23:55:06,225 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/2f62bcef5efb6b2b9f2b4f8143d46f0c 2023-11-19 23:55:06,225 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/7e150658e5024f3f9c25816bd3670929 2023-11-19 23:55:06,227 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/9101d5a9a2db4fd79d2d4bd5ca36f5fe 2023-11-19 23:55:06,227 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5/recovered.edits/90.seqid 2023-11-19 23:55:06,228 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/6f55933bcd938d42ee92cfc06520a6f5 2023-11-19 23:55:06,228 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/bae65117e86641ae9324a04a1ec2e9b9 2023-11-19 23:55:06,230 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/dc06d12bdbed4a48a9e647a03ebbc034 2023-11-19 23:55:06,231 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf1/e31a25b1ee114456839f0f049c3e668f 2023-11-19 23:55:06,233 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/147580c53bc9446da786bb6449fcad57 2023-11-19 23:55:06,234 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/5c49990d6ede4a25ba62c83492438e0a 2023-11-19 23:55:06,235 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/63983129e8d141ee9c75e9d5fc06d9ea 2023-11-19 23:55:06,237 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/9873c766eb824fdb8dddd539fc295576 2023-11-19 23:55:06,238 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/c6766443920742f090e593a23a6f084b 2023-11-19 23:55:06,239 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/cdd504e23a424284938baaf6e26c723b 2023-11-19 23:55:06,240 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/cf2/f558cbf59b0e4b34bb9a5c351618f8bc 2023-11-19 23:55:06,243 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf/recovered.edits/85.seqid 2023-11-19 23:55:06,243 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegionEmptyDaughter/758657921095a87a597d099024cf4cdf 2023-11-19 23:55:06,243 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-11-19 23:55:06,245 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,247 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-11-19 23:55:06,249 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-11-19 23:55:06,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-11-19 23:55:06,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438106250"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:06,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438106250"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:06,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438106250"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:06,252 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:55:06,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 758657921095a87a597d099024cf4cdf, NAME => 'testSplitTableRegionEmptyDaughter,,1700438103066.758657921095a87a597d099024cf4cdf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6f55933bcd938d42ee92cfc06520a6f5, NAME => 'testSplitTableRegionEmptyDaughter,,1700438105038.6f55933bcd938d42ee92cfc06520a6f5.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 2f62bcef5efb6b2b9f2b4f8143d46f0c, NAME => 'testSplitTableRegionEmptyDaughter,71,1700438105038.2f62bcef5efb6b2b9f2b4f8143d46f0c.', STARTKEY => '71', ENDKEY => ''}] 2023-11-19 23:55:06,252 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-11-19 23:55:06,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438106252"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:06,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-11-19 23:55:06,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-11-19 23:55:06,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 48 msec 2023-11-19 23:55:06,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-11-19 23:55:06,312 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-11-19 23:55:06,326 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=380 (was 380), OpenFileDescriptor=869 (was 869), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=120 (was 131), ProcessCount=170 (was 170), AvailableMemoryMB=14846 (was 14850) 2023-11-19 23:55:06,338 INFO [Listener at localhost/42385] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=380, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=120, ProcessCount=170, AvailableMemoryMB=14845 2023-11-19 23:55:06,338 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:55:06,340 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-11-19 23:55:06,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-11-19 23:55:06,341 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-11-19 23:55:06,342 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-11-19 23:55:06,343 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-19 23:55:06,344 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,344 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 empty. 2023-11-19 23:55:06,345 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-11-19 23:55:06,353 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-11-19 23:55:06,354 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 20865392f886f19309f85e8a22235c23, NAME => 'testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 20865392f886f19309f85e8a22235c23, disabling compactions & flushes 2023-11-19 23:55:06,360 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. after waiting 0 ms 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,360 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,360 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:06,361 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-11-19 23:55:06,362 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438106362"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438106362"}]},"ts":"1700438106362"} 2023-11-19 23:55:06,363 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-19 23:55:06,364 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-19 23:55:06,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438106364"}]},"ts":"1700438106364"} 2023-11-19 23:55:06,365 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-11-19 23:55:06,367 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-11-19 23:55:06,368 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-11-19 23:55:06,368 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-11-19 23:55:06,368 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-11-19 23:55:06,368 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-11-19 23:55:06,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, ASSIGN}] 2023-11-19 23:55:06,369 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, ASSIGN 2023-11-19 23:55:06,370 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:55:06,520 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-11-19 23:55:06,521 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=20865392f886f19309f85e8a22235c23, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,521 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438106521"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438106521"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438106521"}]},"ts":"1700438106521"} 2023-11-19 23:55:06,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 20865392f886f19309f85e8a22235c23, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:06,673 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 20865392f886f19309f85e8a22235c23, NAME => 'testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.', STARTKEY => '', ENDKEY => ''} 2023-11-19 23:55:06,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:06,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:06,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,677 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,678 DEBUG [StoreOpener-20865392f886f19309f85e8a22235c23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1 2023-11-19 23:55:06,678 DEBUG [StoreOpener-20865392f886f19309f85e8a22235c23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1 2023-11-19 23:55:06,679 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 20865392f886f19309f85e8a22235c23 columnFamilyName cf1 2023-11-19 23:55:06,679 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] regionserver.HStore(310): Store=20865392f886f19309f85e8a22235c23/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:06,679 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,680 DEBUG [StoreOpener-20865392f886f19309f85e8a22235c23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2 2023-11-19 23:55:06,680 DEBUG [StoreOpener-20865392f886f19309f85e8a22235c23-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2 2023-11-19 23:55:06,680 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 20865392f886f19309f85e8a22235c23 columnFamilyName cf2 2023-11-19 23:55:06,681 INFO [StoreOpener-20865392f886f19309f85e8a22235c23-1] regionserver.HStore(310): Store=20865392f886f19309f85e8a22235c23/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:06,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:06,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:06,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-19 23:55:06,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 20865392f886f19309f85e8a22235c23; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73684596, jitterRate=0.09798604249954224}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:06,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:06,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23., pid=143, masterSystemTime=1700438106673 2023-11-19 23:55:06,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,687 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=20865392f886f19309f85e8a22235c23, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,687 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438106687"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438106687"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438106687"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438106687"}]},"ts":"1700438106687"} 2023-11-19 23:55:06,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=143, resume processing ppid=142 2023-11-19 23:55:06,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 20865392f886f19309f85e8a22235c23, server=jenkins-hbase4.apache.org,36283,1700438000319 in 166 msec 2023-11-19 23:55:06,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=142, resume processing ppid=141 2023-11-19 23:55:06,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, ASSIGN in 322 msec 2023-11-19 23:55:06,691 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-19 23:55:06,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438106691"}]},"ts":"1700438106691"} 2023-11-19 23:55:06,692 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-11-19 23:55:06,695 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-11-19 23:55:06,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 354 msec 2023-11-19 23:55:06,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:06,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:06,848 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:06,848 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:06,848 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:06,849 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:06,849 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,849 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,850 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,850 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,850 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,850 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,851 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,851 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:06,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:06,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:06,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:06,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:06,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:06,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:06,852 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:06,852 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,853 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:06,853 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=120 B heapSize=928 B 2023-11-19 23:55:06,854 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,854 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,854 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:06,855 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,855 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,855 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:06,862 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 2023-11-19 23:55:06,871 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/eff1c10e3f7f427ab69170cae2e28320 2023-11-19 23:55:06,875 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 2023-11-19 23:55:06,878 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:55:06,879 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/eff1c10e3f7f427ab69170cae2e28320 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320 2023-11-19 23:55:06,882 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:55:06,882 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 29ms, sequenceid=6, compaction requested=false 2023-11-19 23:55:06,882 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-11-19 23:55:06,883 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:06,883 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:06,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:06,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:06,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:06,883 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:06,885 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:06,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:06,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:06,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:06,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b104b6b[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:06,887 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:06,887 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:06,888 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:06,888 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,889 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:06,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:06,888 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:06,889 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:06,890 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:06,890 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:06,891 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:06,891 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:06,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:06,894 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:06,894 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:06,894 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:06,894 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:06,894 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,895 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:06,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,896 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,896 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:06,896 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:06,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:06,897 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:06,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,899 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:06,899 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:06,899 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:06,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:06,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,903 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:06,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:06,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:06,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:06,904 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,004 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:07,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:07,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:07,015 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:07,015 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,015 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:07,016 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:07,016 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,016 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,020 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,021 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,021 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,021 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,022 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,022 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:07,023 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:07,023 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,023 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:07,023 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:07,024 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,025 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,025 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,025 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,025 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,025 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,033 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/82dc8a706f3840a291619788858adb45 2023-11-19 23:55:07,042 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 2023-11-19 23:55:07,045 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/82dc8a706f3840a291619788858adb45 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45 2023-11-19 23:55:07,048 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:55:07,049 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 2023-11-19 23:55:07,051 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5, entries=10, sequenceid=19, filesize=5.0 K 2023-11-19 23:55:07,052 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 29ms, sequenceid=19, compaction requested=false 2023-11-19 23:55:07,053 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:07,053 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,055 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,055 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,056 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,056 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43025209[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:07,057 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:07,058 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,059 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,059 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:07,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,059 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,061 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,061 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:07,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:07,064 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:07,064 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:07,064 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:07,064 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:07,065 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:07,067 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,067 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,067 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,067 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,067 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,068 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,069 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,070 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,072 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,072 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,072 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,075 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:07,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,075 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,076 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:07,076 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,076 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,076 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,077 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,177 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:07,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:07,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:07,189 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:07,189 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,189 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:07,189 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:07,189 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,189 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,190 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,190 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,190 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,190 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,191 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,191 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,192 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,192 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:07,193 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:07,193 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,193 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:07,193 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:07,194 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,194 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,195 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,195 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,195 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,201 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/e3a57b1537c745aab1f68ce0283af1c2 2023-11-19 23:55:07,211 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/48edfa4ce2e045968c94398d1f272ea7 2023-11-19 23:55:07,214 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/e3a57b1537c745aab1f68ce0283af1c2 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2 2023-11-19 23:55:07,217 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:55:07,218 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/48edfa4ce2e045968c94398d1f272ea7 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7 2023-11-19 23:55:07,221 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7, entries=10, sequenceid=32, filesize=5.0 K 2023-11-19 23:55:07,222 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 29ms, sequenceid=32, compaction requested=true 2023-11-19 23:55:07,222 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:07,222 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:07,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:07,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,222 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,224 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,225 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,225 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66d33055[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:07,226 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:07,226 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,227 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,227 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,227 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:07,228 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,228 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:07,229 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,229 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,229 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:07,237 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:07,237 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:07,238 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:07,238 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:07,238 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,239 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,240 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,241 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,242 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,243 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,243 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:07,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,246 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,246 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:07,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,247 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,247 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:07,247 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,247 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,247 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,347 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:07,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:07,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:07,358 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:07,358 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,358 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:07,358 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:07,359 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,359 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,360 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,360 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,360 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,362 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,362 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:07,362 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:07,362 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,362 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:07,362 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,365 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,365 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,365 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,365 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,365 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,370 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/245fefc2953341959ac21c4ad7d32fdd 2023-11-19 23:55:07,379 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/e1d544ed312a4900a160a605dc8e68ee 2023-11-19 23:55:07,382 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/245fefc2953341959ac21c4ad7d32fdd as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd 2023-11-19 23:55:07,385 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:55:07,386 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/e1d544ed312a4900a160a605dc8e68ee as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee 2023-11-19 23:55:07,389 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee, entries=10, sequenceid=45, filesize=5.0 K 2023-11-19 23:55:07,390 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 28ms, sequenceid=45, compaction requested=true 2023-11-19 23:55:07,390 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:07,390 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,390 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:07,390 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:07,390 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,390 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,390 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,391 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,391 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,392 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,392 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:07,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73f7d709[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:07,394 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:07,394 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,395 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,395 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:07,396 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:07,398 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:07,401 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:07,401 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:07,401 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:07,401 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:07,401 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,403 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,404 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,405 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,408 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,410 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,410 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,410 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,414 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,413 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:07,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,414 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,414 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:07,415 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:07,515 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:07,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:07,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:07,526 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:07,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:07,526 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:07,527 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,527 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,528 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,528 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,528 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,528 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,529 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,529 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,533 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:07,533 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:07,534 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,534 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:07,534 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:07,537 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,538 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,538 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,538 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,538 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,538 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,546 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/740f969bc15b4ab49e153f3c26466edb 2023-11-19 23:55:07,556 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b 2023-11-19 23:55:07,559 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/740f969bc15b4ab49e153f3c26466edb as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb 2023-11-19 23:55:07,562 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:55:07,563 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b 2023-11-19 23:55:07,565 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b, entries=10, sequenceid=58, filesize=5.0 K 2023-11-19 23:55:07,566 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 32ms, sequenceid=58, compaction requested=true 2023-11-19 23:55:07,566 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:07,566 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,566 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:07,566 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:07,566 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,566 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,566 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,569 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,569 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,569 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,569 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a85910f[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:07,571 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:07,571 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,573 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:07,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,573 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,574 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,574 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,575 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,575 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,575 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:07,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:07,577 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:07,578 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:07,578 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:07,578 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:07,578 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,580 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,580 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,581 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,583 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,584 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:07,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,586 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:07,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:07,586 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,587 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,668 ERROR [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid c1e6d43a-1c05-42be-a588-44ff607a74aa) service to localhost/127.0.0.1:45125 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:55:07,687 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-19 23:55:07,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-11-19 23:55:07,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-11-19 23:55:07,699 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-11-19 23:55:07,699 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,699 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-11-19 23:55:07,699 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-11-19 23:55:07,699 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,699 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,700 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,700 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,700 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,700 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,701 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,701 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-11-19 23:55:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-11-19 23:55:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,703 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-19 23:55:07,703 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. started... 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34057,1700438000374' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,703 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-11-19 23:55:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39613,1700438000411' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,704 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,704 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,711 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/89764fd3eb154045aef3e893e7c82797 2023-11-19 23:55:07,720 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/5f53077d908b4e0ab6619467c3421c9f 2023-11-19 23:55:07,723 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/89764fd3eb154045aef3e893e7c82797 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797 2023-11-19 23:55:07,726 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:55:07,726 ERROR [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid 92965f72-b1d2-4a0b-81fa-7e4974137a5a) service to localhost/127.0.0.1:45125 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-11-19 23:55:07,727 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/5f53077d908b4e0ab6619467c3421c9f as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f 2023-11-19 23:55:07,729 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f, entries=10, sequenceid=71, filesize=5.0 K 2023-11-19 23:55:07,730 INFO [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 27ms, sequenceid=71, compaction requested=true 2023-11-19 23:55:07,730 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:07,730 DEBUG [rs(jenkins-hbase4.apache.org,36283,1700438000319)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:07,730 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-19 23:55:07,730 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-19 23:55:07,730 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,730 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-11-19 23:55:07,730 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-11-19 23:55:07,733 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,733 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,733 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,733 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-19 23:55:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36283,1700438000319' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-11-19 23:55:07,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b46004c[Count = 0] remaining members to acquire global barrier 2023-11-19 23:55:07,735 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-11-19 23:55:07,735 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,736 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-19 23:55:07,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,736 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,736 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39613,1700438000411' in zk 2023-11-19 23:55:07,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,737 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36283,1700438000319' in zk 2023-11-19 23:55:07,736 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-11-19 23:55:07,737 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-11-19 23:55:07,737 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34057,1700438000374' in zk 2023-11-19 23:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,39613,1700438000411' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,739 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-11-19 23:55:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,34057,1700438000374' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,739 DEBUG [member: 'jenkins-hbase4.apache.org,36283,1700438000319' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-11-19 23:55:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,36283,1700438000319': 2023-11-19 23:55:07,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36283,1700438000319' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-11-19 23:55:07,742 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-11-19 23:55:07,742 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-19 23:55:07,742 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-19 23:55:07,742 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-11-19 23:55:07,742 INFO [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-19 23:55:07,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-19 23:55:07,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,746 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-19 23:55:07,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,747 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-19 23:55:07,747 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,748 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-19 23:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,749 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-19 23:55:07,749 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-11-19 23:55:07,750 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,752 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,752 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,752 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,752 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,753 DEBUG [(jenkins-hbase4.apache.org,45383,1700437998977)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-19 23:55:07,752 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-11-19 23:55:07,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-19 23:55:07,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-11-19 23:55:07,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-11-19 23:55:07,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-19 23:55:07,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-19 23:55:07,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:07,753 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-19 23:55:07,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-19 23:55:07,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-19 23:55:07,754 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,754 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,754 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:07,754 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-11-19 23:55:07,754 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-11-19 23:55:07,854 DEBUG [Listener at localhost/42385] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-19 23:55:07,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-19 23:55:07,861 DEBUG [Listener at localhost/42385] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,862 DEBUG [Listener at localhost/42385] procedure2.ProcedureExecutor(1028): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=20865392f886f19309f85e8a22235c23, daughterA=a97f8cd4bef8ae30620051e7f18876d1, daughterB=86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:07,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=20865392f886f19309f85e8a22235c23, daughterA=a97f8cd4bef8ae30620051e7f18876d1, daughterB=86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:07,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=20865392f886f19309f85e8a22235c23, daughterA=a97f8cd4bef8ae30620051e7f18876d1, daughterB=86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:07,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=20865392f886f19309f85e8a22235c23, daughterA=a97f8cd4bef8ae30620051e7f18876d1, daughterB=86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:07,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, UNASSIGN}] 2023-11-19 23:55:07,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, UNASSIGN 2023-11-19 23:55:07,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=20865392f886f19309f85e8a22235c23, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:07,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438107865"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438107865"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438107865"}]},"ts":"1700438107865"} 2023-11-19 23:55:07,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 20865392f886f19309f85e8a22235c23, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:08,017 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:08,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 20865392f886f19309f85e8a22235c23, disabling compactions & flushes 2023-11-19 23:55:08,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:08,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:08,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. after waiting 0 ms 2023-11-19 23:55:08,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:08,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 20865392f886f19309f85e8a22235c23 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-11-19 23:55:08,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/6d8db07f70a7446cbb669ef2084cf088 2023-11-19 23:55:08,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/45417d09aed1455ab29c577a100a447e 2023-11-19 23:55:08,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf1/6d8db07f70a7446cbb669ef2084cf088 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088 2023-11-19 23:55:08,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:55:08,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/.tmp/cf2/45417d09aed1455ab29c577a100a447e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e 2023-11-19 23:55:08,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e, entries=8, sequenceid=82, filesize=5.0 K 2023-11-19 23:55:08,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 20865392f886f19309f85e8a22235c23 in 27ms, sequenceid=82, compaction requested=true 2023-11-19 23:55:08,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-11-19 23:55:08,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:08,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. 2023-11-19 23:55:08,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 20865392f886f19309f85e8a22235c23: 2023-11-19 23:55:08,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=20865392f886f19309f85e8a22235c23, regionState=CLOSED 2023-11-19 23:55:08,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438108050"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438108050"}]},"ts":"1700438108050"} 2023-11-19 23:55:08,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=146, resume processing ppid=145 2023-11-19 23:55:08,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 20865392f886f19309f85e8a22235c23, server=jenkins-hbase4.apache.org,36283,1700438000319 in 185 msec 2023-11-19 23:55:08,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=145, resume processing ppid=144 2023-11-19 23:55:08,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=20865392f886f19309f85e8a22235c23, UNASSIGN in 188 msec 2023-11-19 23:55:08,061 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=20865392f886f19309f85e8a22235c23, threads=14 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,061 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,062 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,062 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,062 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,062 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,062 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,121 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,129 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,129 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,144 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,160 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,163 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,170 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,170 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,172 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,172 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,175 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,521 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,522 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,919 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45 for region: 20865392f886f19309f85e8a22235c23 2023-11-19 23:55:08,919 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 20865392f886f19309f85e8a22235c23 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-11-19 23:55:09,350 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:09,351 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-11-19 23:55:09,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1700438109352"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1700438109352"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1700438109352"}]},"ts":"1700438109352"} 2023-11-19 23:55:09,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109352"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438109352"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438109352"}]},"ts":"1700438109352"} 2023-11-19 23:55:09,352 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109352"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438109352"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438109352"}]},"ts":"1700438109352"} 2023-11-19 23:55:09,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, ASSIGN}] 2023-11-19 23:55:09,355 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, ASSIGN 2023-11-19 23:55:09,356 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:55:09,356 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, ASSIGN 2023-11-19 23:55:09,357 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36283,1700438000319; forceNewPlan=false, retain=false 2023-11-19 23:55:09,506 INFO [jenkins-hbase4:45383] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-11-19 23:55:09,508 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=86bd2f7e07afd96623b8409c1251c938, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109508"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438109508"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438109508"}]},"ts":"1700438109508"} 2023-11-19 23:55:09,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 86bd2f7e07afd96623b8409c1251c938, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:09,510 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=a97f8cd4bef8ae30620051e7f18876d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109510"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438109510"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438109510"}]},"ts":"1700438109510"} 2023-11-19 23:55:09,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure a97f8cd4bef8ae30620051e7f18876d1, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:09,660 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 86bd2f7e07afd96623b8409c1251c938, NAME => 'testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.', STARTKEY => '41', ENDKEY => ''} 2023-11-19 23:55:09,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:09,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:09,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,664 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,665 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1 2023-11-19 23:55:09,665 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1 2023-11-19 23:55:09,665 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86bd2f7e07afd96623b8409c1251c938 columnFamilyName cf1 2023-11-19 23:55:09,677 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-top 2023-11-19 23:55:09,680 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-top 2023-11-19 23:55:09,683 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-top 2023-11-19 23:55:09,685 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-top 2023-11-19 23:55:09,688 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-top 2023-11-19 23:55:09,691 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-top 2023-11-19 23:55:09,694 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-top 2023-11-19 23:55:09,694 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(310): Store=86bd2f7e07afd96623b8409c1251c938/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:09,694 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,695 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2 2023-11-19 23:55:09,695 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2 2023-11-19 23:55:09,695 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86bd2f7e07afd96623b8409c1251c938 columnFamilyName cf2 2023-11-19 23:55:09,705 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-top 2023-11-19 23:55:09,708 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-top 2023-11-19 23:55:09,710 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-top 2023-11-19 23:55:09,713 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-top 2023-11-19 23:55:09,716 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-top 2023-11-19 23:55:09,719 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-top 2023-11-19 23:55:09,722 DEBUG [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-top 2023-11-19 23:55:09,722 INFO [StoreOpener-86bd2f7e07afd96623b8409c1251c938-1] regionserver.HStore(310): Store=86bd2f7e07afd96623b8409c1251c938/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:09,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:09,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:09,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 86bd2f7e07afd96623b8409c1251c938; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72360661, jitterRate=0.07825787365436554}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:09,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 86bd2f7e07afd96623b8409c1251c938: 2023-11-19 23:55:09,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938., pid=149, masterSystemTime=1700438109660 2023-11-19 23:55:09,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,729 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:09,729 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:09,729 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,729 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,729 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): 86bd2f7e07afd96623b8409c1251c938/cf1 is initiating minor compaction (all files) 2023-11-19 23:55:09,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,729 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 86bd2f7e07afd96623b8409c1251c938/cf1 in testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,729 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): 86bd2f7e07afd96623b8409c1251c938/cf2 is initiating minor compaction (all files) 2023-11-19 23:55:09,729 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 86bd2f7e07afd96623b8409c1251c938/cf2 in testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,730 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/.tmp, totalSize=34.9 K 2023-11-19 23:55:09,730 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-top] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/.tmp, totalSize=34.9 K 2023-11-19 23:55:09,730 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438106843 2023-11-19 23:55:09,730 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1700438106843 2023-11-19 23:55:09,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:09,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,731 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438107005 2023-11-19 23:55:09,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a97f8cd4bef8ae30620051e7f18876d1, NAME => 'testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.', STARTKEY => '', ENDKEY => '41'} 2023-11-19 23:55:09,731 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=86bd2f7e07afd96623b8409c1251c938, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-11-19 23:55:09,731 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1700438107005 2023-11-19 23:55:09,731 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109731"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438109731"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438109731"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438109731"}]},"ts":"1700438109731"} 2023-11-19 23:55:09,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-19 23:55:09,731 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438107178 2023-11-19 23:55:09,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,731 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1700438107178 2023-11-19 23:55:09,732 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438107348 2023-11-19 23:55:09,732 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1700438107348 2023-11-19 23:55:09,732 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438107516 2023-11-19 23:55:09,733 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1700438107516 2023-11-19 23:55:09,733 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438107688 2023-11-19 23:55:09,733 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,733 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1700438107688 2023-11-19 23:55:09,733 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438107855 2023-11-19 23:55:09,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=149, resume processing ppid=148 2023-11-19 23:55:09,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 86bd2f7e07afd96623b8409c1251c938, server=jenkins-hbase4.apache.org,36283,1700438000319 in 223 msec 2023-11-19 23:55:09,734 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1700438107855 2023-11-19 23:55:09,734 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1 2023-11-19 23:55:09,734 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1 2023-11-19 23:55:09,735 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a97f8cd4bef8ae30620051e7f18876d1 columnFamilyName cf1 2023-11-19 23:55:09,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, ASSIGN in 379 msec 2023-11-19 23:55:09,745 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): 86bd2f7e07afd96623b8409c1251c938#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:09,745 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): 86bd2f7e07afd96623b8409c1251c938#cf2#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:09,749 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-bottom 2023-11-19 23:55:09,760 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-bottom 2023-11-19 23:55:09,764 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/.tmp/cf2/7a7513a18d374a4985b45d08b5944234 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/7a7513a18d374a4985b45d08b5944234 2023-11-19 23:55:09,765 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-bottom 2023-11-19 23:55:09,766 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/.tmp/cf1/dbfccd92310d43cd95fd3e2317aec38d as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/dbfccd92310d43cd95fd3e2317aec38d 2023-11-19 23:55:09,768 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-bottom 2023-11-19 23:55:09,769 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 86bd2f7e07afd96623b8409c1251c938/cf2 of 86bd2f7e07afd96623b8409c1251c938 into 7a7513a18d374a4985b45d08b5944234(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:09,769 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 86bd2f7e07afd96623b8409c1251c938: 2023-11-19 23:55:09,769 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938., storeName=86bd2f7e07afd96623b8409c1251c938/cf2, priority=9, startTime=1700438109729; duration=0sec 2023-11-19 23:55:09,769 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,770 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 86bd2f7e07afd96623b8409c1251c938/cf1 of 86bd2f7e07afd96623b8409c1251c938 into dbfccd92310d43cd95fd3e2317aec38d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:09,770 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 86bd2f7e07afd96623b8409c1251c938: 2023-11-19 23:55:09,770 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938., storeName=86bd2f7e07afd96623b8409c1251c938/cf1, priority=9, startTime=1700438109728; duration=0sec 2023-11-19 23:55:09,770 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,772 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-bottom 2023-11-19 23:55:09,775 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-bottom 2023-11-19 23:55:09,777 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-bottom 2023-11-19 23:55:09,778 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(310): Store=a97f8cd4bef8ae30620051e7f18876d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:09,778 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,778 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2 2023-11-19 23:55:09,778 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2 2023-11-19 23:55:09,779 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a97f8cd4bef8ae30620051e7f18876d1 columnFamilyName cf2 2023-11-19 23:55:09,789 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-bottom 2023-11-19 23:55:09,792 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-bottom 2023-11-19 23:55:09,794 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-bottom 2023-11-19 23:55:09,797 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-bottom 2023-11-19 23:55:09,799 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-bottom 2023-11-19 23:55:09,802 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-bottom 2023-11-19 23:55:09,804 DEBUG [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(539): loaded hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-bottom 2023-11-19 23:55:09,805 INFO [StoreOpener-a97f8cd4bef8ae30620051e7f18876d1-1] regionserver.HStore(310): Store=a97f8cd4bef8ae30620051e7f18876d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-19 23:55:09,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-19 23:55:09,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:09,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a97f8cd4bef8ae30620051e7f18876d1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72736850, jitterRate=0.0838635265827179}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-19 23:55:09,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a97f8cd4bef8ae30620051e7f18876d1: 2023-11-19 23:55:09,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1., pid=150, masterSystemTime=1700438109660 2023-11-19 23:55:09,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-19 23:55:09,810 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:09,810 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,810 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1912): a97f8cd4bef8ae30620051e7f18876d1/cf1 is initiating minor compaction (all files) 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a97f8cd4bef8ae30620051e7f18876d1/cf1 in testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,810 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1912): a97f8cd4bef8ae30620051e7f18876d1/cf2 is initiating minor compaction (all files) 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a97f8cd4bef8ae30620051e7f18876d1/cf2 in testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/.tmp, totalSize=34.9 K 2023-11-19 23:55:09,810 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-bottom] into tmpdir=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/.tmp, totalSize=34.9 K 2023-11-19 23:55:09,811 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438106843 2023-11-19 23:55:09,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:09,811 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1700438106843 2023-11-19 23:55:09,811 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438107005 2023-11-19 23:55:09,811 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1700438107005 2023-11-19 23:55:09,812 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=a97f8cd4bef8ae30620051e7f18876d1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,812 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438109811"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1700438109811"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1700438109811"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1700438109811"}]},"ts":"1700438109811"} 2023-11-19 23:55:09,812 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438107178 2023-11-19 23:55:09,812 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1700438107178 2023-11-19 23:55:09,812 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438107348 2023-11-19 23:55:09,812 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1700438107348 2023-11-19 23:55:09,813 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438107516 2023-11-19 23:55:09,813 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1700438107516 2023-11-19 23:55:09,813 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438107688 2023-11-19 23:55:09,813 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1700438107688 2023-11-19 23:55:09,813 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] compactions.Compactor(207): Compacting 6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438107855 2023-11-19 23:55:09,814 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] compactions.Compactor(207): Compacting 45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1700438107855 2023-11-19 23:55:09,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=150, resume processing ppid=147 2023-11-19 23:55:09,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure a97f8cd4bef8ae30620051e7f18876d1, server=jenkins-hbase4.apache.org,36283,1700438000319 in 302 msec 2023-11-19 23:55:09,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=147, resume processing ppid=144 2023-11-19 23:55:09,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, ASSIGN in 459 msec 2023-11-19 23:55:09,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=20865392f886f19309f85e8a22235c23, daughterA=a97f8cd4bef8ae30620051e7f18876d1, daughterB=86bd2f7e07afd96623b8409c1251c938 in 1.9550 sec 2023-11-19 23:55:09,822 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] throttle.PressureAwareThroughputController(145): a97f8cd4bef8ae30620051e7f18876d1#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:09,822 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] throttle.PressureAwareThroughputController(145): a97f8cd4bef8ae30620051e7f18876d1#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-19 23:55:09,831 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/.tmp/cf2/bd74bf96ab304275b92e29113d762214 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/bd74bf96ab304275b92e29113d762214 2023-11-19 23:55:09,834 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a97f8cd4bef8ae30620051e7f18876d1/cf2 of a97f8cd4bef8ae30620051e7f18876d1 into bd74bf96ab304275b92e29113d762214(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:09,834 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a97f8cd4bef8ae30620051e7f18876d1: 2023-11-19 23:55:09,834 INFO [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1., storeName=a97f8cd4bef8ae30620051e7f18876d1/cf2, priority=9, startTime=1700438109810; duration=0sec 2023-11-19 23:55:09,834 DEBUG [RS:0;jenkins-hbase4:36283-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:09,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36283] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:48854 deadline: 1700438169863, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23. is not online on jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:09,970 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:55:09,976 INFO [Listener at localhost/42385] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-11-19 23:55:09,978 DEBUG [Listener at localhost/42385] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-11-19 23:55:09,992 WARN [Listener at localhost/42385] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-11-19 23:55:09,994 INFO [Listener at localhost/42385] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-11-19 23:55:09,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-11-19 23:55:09,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-11-19 23:55:09,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-11-19 23:55:09,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438109997"}]},"ts":"1700438109997"} 2023-11-19 23:55:09,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-11-19 23:55:09,999 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-11-19 23:55:10,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, UNASSIGN}] 2023-11-19 23:55:10,000 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, UNASSIGN 2023-11-19 23:55:10,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=86bd2f7e07afd96623b8409c1251c938, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438110001"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438110001"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438110001"}]},"ts":"1700438110001"} 2023-11-19 23:55:10,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 86bd2f7e07afd96623b8409c1251c938, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:10,003 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, UNASSIGN 2023-11-19 23:55:10,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=a97f8cd4bef8ae30620051e7f18876d1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,004 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438110004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1700438110004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1700438110004"}]},"ts":"1700438110004"} 2023-11-19 23:55:10,004 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45383] assignment.AssignmentManager(1312): Unable to acquire lock for regionNode state=CLOSING, location=jenkins-hbase4.apache.org,36283,1700438000319, table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1. It is likely that another thread is currently holding the lock. To avoid deadlock, skip execution for now. 2023-11-19 23:55:10,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure a97f8cd4bef8ae30620051e7f18876d1, server=jenkins-hbase4.apache.org,36283,1700438000319}] 2023-11-19 23:55:10,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-11-19 23:55:10,153 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:10,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a97f8cd4bef8ae30620051e7f18876d1, disabling compactions & flushes 2023-11-19 23:55:10,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:10,232 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/.tmp/cf1/f9b1db609ee44f6e8fe0e5ce0d0a5a0e as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/f9b1db609ee44f6e8fe0e5ce0d0a5a0e 2023-11-19 23:55:10,235 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a97f8cd4bef8ae30620051e7f18876d1/cf1 of a97f8cd4bef8ae30620051e7f18876d1 into f9b1db609ee44f6e8fe0e5ce0d0a5a0e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-19 23:55:10,235 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a97f8cd4bef8ae30620051e7f18876d1: 2023-11-19 23:55:10,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:10,235 INFO [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1., storeName=a97f8cd4bef8ae30620051e7f18876d1/cf1, priority=9, startTime=1700438109809; duration=0sec 2023-11-19 23:55:10,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:10,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. after waiting 0 ms 2023-11-19 23:55:10,235 DEBUG [RS:0;jenkins-hbase4:36283-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-19 23:55:10,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:10,236 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-bottom] to archive 2023-11-19 23:55:10,236 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:10,238 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,239 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,239 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,240 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,241 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,242 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,243 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,244 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-bottom, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-bottom] to archive 2023-11-19 23:55:10,244 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:10,245 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,246 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,247 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,248 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,249 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,250 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,250 DEBUG [StoreCloser-testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:55:10,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:10,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1. 2023-11-19 23:55:10,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a97f8cd4bef8ae30620051e7f18876d1: 2023-11-19 23:55:10,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:10,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:10,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 86bd2f7e07afd96623b8409c1251c938, disabling compactions & flushes 2023-11-19 23:55:10,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:10,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:10,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. after waiting 0 ms 2023-11-19 23:55:10,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:10,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=a97f8cd4bef8ae30620051e7f18876d1, regionState=CLOSED 2023-11-19 23:55:10,255 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438110255"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438110255"}]},"ts":"1700438110255"} 2023-11-19 23:55:10,256 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088-top] to archive 2023-11-19 23:55:10,257 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:10,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=155, resume processing ppid=152 2023-11-19 23:55:10,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure a97f8cd4bef8ae30620051e7f18876d1, server=jenkins-hbase4.apache.org,36283,1700438000319 in 251 msec 2023-11-19 23:55:10,258 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/7abf13c1b3c74c6b8d9dac5f1b796777.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=a97f8cd4bef8ae30620051e7f18876d1, UNASSIGN in 258 msec 2023-11-19 23:55:10,259 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/82dc8a706f3840a291619788858adb45.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,260 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/e3a57b1537c745aab1f68ce0283af1c2.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,261 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/245fefc2953341959ac21c4ad7d32fdd.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,262 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/740f969bc15b4ab49e153f3c26466edb.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,263 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/89764fd3eb154045aef3e893e7c82797.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,264 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/6d8db07f70a7446cbb669ef2084cf088.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,265 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f-top, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23->hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e-top] to archive 2023-11-19 23:55:10,266 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-19 23:55:10,267 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/eff1c10e3f7f427ab69170cae2e28320.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,268 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/3c24ee9eb052440b89bbab5da2a0a2e5.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,268 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/48edfa4ce2e045968c94398d1f272ea7.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,269 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/e1d544ed312a4900a160a605dc8e68ee.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,270 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,271 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/5f53077d908b4e0ab6619467c3421c9f.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,272 DEBUG [StoreCloser-testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/45417d09aed1455ab29c577a100a447e.20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-11-19 23:55:10,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:10,275 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938. 2023-11-19 23:55:10,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 86bd2f7e07afd96623b8409c1251c938: 2023-11-19 23:55:10,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:10,276 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=86bd2f7e07afd96623b8409c1251c938, regionState=CLOSED 2023-11-19 23:55:10,277 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1700438110276"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1700438110276"}]},"ts":"1700438110276"} 2023-11-19 23:55:10,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=154, resume processing ppid=153 2023-11-19 23:55:10,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 86bd2f7e07afd96623b8409c1251c938, server=jenkins-hbase4.apache.org,36283,1700438000319 in 275 msec 2023-11-19 23:55:10,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=153, resume processing ppid=151 2023-11-19 23:55:10,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=86bd2f7e07afd96623b8409c1251c938, UNASSIGN in 278 msec 2023-11-19 23:55:10,280 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1700438110280"}]},"ts":"1700438110280"} 2023-11-19 23:55:10,281 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-11-19 23:55:10,282 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-11-19 23:55:10,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 288 msec 2023-11-19 23:55:10,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-11-19 23:55:10,298 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-11-19 23:55:10,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-11-19 23:55:10,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] procedure2.ProcedureExecutor(1028): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,301 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,301 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-11-19 23:55:10,304 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,304 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:10,304 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:10,306 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/recovered.edits] 2023-11-19 23:55:10,306 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/recovered.edits] 2023-11-19 23:55:10,306 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2, FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/recovered.edits] 2023-11-19 23:55:10,308 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/f9b1db609ee44f6e8fe0e5ce0d0a5a0e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf1/f9b1db609ee44f6e8fe0e5ce0d0a5a0e 2023-11-19 23:55:10,309 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/dbfccd92310d43cd95fd3e2317aec38d to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf1/dbfccd92310d43cd95fd3e2317aec38d 2023-11-19 23:55:10,309 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/245fefc2953341959ac21c4ad7d32fdd 2023-11-19 23:55:10,311 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/bd74bf96ab304275b92e29113d762214 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/cf2/bd74bf96ab304275b92e29113d762214 2023-11-19 23:55:10,311 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/6d8db07f70a7446cbb669ef2084cf088 2023-11-19 23:55:10,311 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/7a7513a18d374a4985b45d08b5944234 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/cf2/7a7513a18d374a4985b45d08b5944234 2023-11-19 23:55:10,312 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/740f969bc15b4ab49e153f3c26466edb 2023-11-19 23:55:10,313 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1/recovered.edits/90.seqid 2023-11-19 23:55:10,313 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/7abf13c1b3c74c6b8d9dac5f1b796777 2023-11-19 23:55:10,314 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/a97f8cd4bef8ae30620051e7f18876d1 2023-11-19 23:55:10,314 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/recovered.edits/90.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938/recovered.edits/90.seqid 2023-11-19 23:55:10,315 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/82dc8a706f3840a291619788858adb45 2023-11-19 23:55:10,315 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/86bd2f7e07afd96623b8409c1251c938 2023-11-19 23:55:10,316 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/89764fd3eb154045aef3e893e7c82797 2023-11-19 23:55:10,316 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf1/e3a57b1537c745aab1f68ce0283af1c2 2023-11-19 23:55:10,318 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/3c24ee9eb052440b89bbab5da2a0a2e5 2023-11-19 23:55:10,319 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/45417d09aed1455ab29c577a100a447e 2023-11-19 23:55:10,320 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/48edfa4ce2e045968c94398d1f272ea7 2023-11-19 23:55:10,321 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5acf4c60ad5d4c36b00d2f29f27ba37b 2023-11-19 23:55:10,322 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/5f53077d908b4e0ab6619467c3421c9f 2023-11-19 23:55:10,323 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/e1d544ed312a4900a160a605dc8e68ee 2023-11-19 23:55:10,324 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320 to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/cf2/eff1c10e3f7f427ab69170cae2e28320 2023-11-19 23:55:10,326 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/recovered.edits/85.seqid to hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/archive/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23/recovered.edits/85.seqid 2023-11-19 23:55:10,326 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/.tmp/data/default/testSplitTableRegion/20865392f886f19309f85e8a22235c23 2023-11-19 23:55:10,326 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-11-19 23:55:10,328 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,329 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-11-19 23:55:10,330 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-11-19 23:55:10,331 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,331 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-11-19 23:55:10,331 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438110331"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:10,331 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438110331"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:10,331 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1700438110331"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:10,332 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-19 23:55:10,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 20865392f886f19309f85e8a22235c23, NAME => 'testSplitTableRegion,,1700438106340.20865392f886f19309f85e8a22235c23.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a97f8cd4bef8ae30620051e7f18876d1, NAME => 'testSplitTableRegion,,1700438107861.a97f8cd4bef8ae30620051e7f18876d1.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 86bd2f7e07afd96623b8409c1251c938, NAME => 'testSplitTableRegion,41,1700438107861.86bd2f7e07afd96623b8409c1251c938.', STARTKEY => '41', ENDKEY => ''}] 2023-11-19 23:55:10,333 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-11-19 23:55:10,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1700438110333"}]},"ts":"9223372036854775807"} 2023-11-19 23:55:10,334 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-11-19 23:55:10,337 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-11-19 23:55:10,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 38 msec 2023-11-19 23:55:10,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45383] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-11-19 23:55:10,402 INFO [Listener at localhost/42385] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-11-19 23:55:10,417 INFO [Listener at localhost/42385] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=380 (was 380), OpenFileDescriptor=881 (was 869) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=143 (was 120) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=14839 (was 14845) 2023-11-19 23:55:10,418 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-11-19 23:55:10,418 INFO [Listener at localhost/42385] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-19 23:55:10,418 DEBUG [Listener at localhost/42385] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7b833cd0 to 127.0.0.1:52135 2023-11-19 23:55:10,418 DEBUG [Listener at localhost/42385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,419 DEBUG [Listener at localhost/42385] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-11-19 23:55:10,419 DEBUG [Listener at localhost/42385] util.JVMClusterUtil(257): Found active master hash=934827988, stopped=false 2023-11-19 23:55:10,419 INFO [Listener at localhost/42385] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:55:10,420 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-19 23:55:10,420 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-19 23:55:10,420 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-19 23:55:10,420 INFO [Listener at localhost/42385] procedure2.ProcedureExecutor(628): Stopping 2023-11-19 23:55:10,420 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-19 23:55:10,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:55:10,420 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:55:10,421 DEBUG [Listener at localhost/42385] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ca9643d to 127.0.0.1:52135 2023-11-19 23:55:10,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:55:10,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:55:10,421 DEBUG [Listener at localhost/42385] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-19 23:55:10,421 INFO [Listener at localhost/42385] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36283,1700438000319' ***** 2023-11-19 23:55:10,422 INFO [Listener at localhost/42385] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-11-19 23:55:10,422 INFO [Listener at localhost/42385] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34057,1700438000374' ***** 2023-11-19 23:55:10,422 INFO [Listener at localhost/42385] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-11-19 23:55:10,422 INFO [Listener at localhost/42385] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39613,1700438000411' ***** 2023-11-19 23:55:10,422 INFO [Listener at localhost/42385] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-11-19 23:55:10,422 INFO [RS:2;jenkins-hbase4:39613] regionserver.HeapMemoryManager(220): Stopping 2023-11-19 23:55:10,422 INFO [RS:0;jenkins-hbase4:36283] regionserver.HeapMemoryManager(220): Stopping 2023-11-19 23:55:10,422 INFO [RS:1;jenkins-hbase4:34057] regionserver.HeapMemoryManager(220): Stopping 2023-11-19 23:55:10,422 INFO [RS:2;jenkins-hbase4:39613] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-19 23:55:10,422 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-19 23:55:10,422 INFO [RS:2;jenkins-hbase4:39613] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-19 23:55:10,422 INFO [RS:0;jenkins-hbase4:36283] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-19 23:55:10,423 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(3305): Received CLOSE for 329994419c4b206406658a465b604c17 2023-11-19 23:55:10,423 INFO [RS:0;jenkins-hbase4:36283] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-19 23:55:10,423 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,423 DEBUG [RS:0;jenkins-hbase4:36283] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x029f2d0e to 127.0.0.1:52135 2023-11-19 23:55:10,423 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:10,422 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-19 23:55:10,422 INFO [RS:1;jenkins-hbase4:34057] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-19 23:55:10,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 329994419c4b206406658a465b604c17, disabling compactions & flushes 2023-11-19 23:55:10,423 INFO [RS:1;jenkins-hbase4:34057] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-19 23:55:10,423 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:10,423 DEBUG [RS:2;jenkins-hbase4:39613] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18f7687e to 127.0.0.1:52135 2023-11-19 23:55:10,423 DEBUG [RS:0;jenkins-hbase4:36283] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,423 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-19 23:55:10,423 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36283,1700438000319; all regions closed. 2023-11-19 23:55:10,423 DEBUG [RS:2;jenkins-hbase4:39613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,423 DEBUG [RS:1;jenkins-hbase4:34057] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2b645a18 to 127.0.0.1:52135 2023-11-19 23:55:10,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:55:10,424 DEBUG [RS:1;jenkins-hbase4:34057] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,424 INFO [RS:2;jenkins-hbase4:39613] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-19 23:55:10,424 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34057,1700438000374; all regions closed. 2023-11-19 23:55:10,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:55:10,424 INFO [RS:2;jenkins-hbase4:39613] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-19 23:55:10,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. after waiting 0 ms 2023-11-19 23:55:10,425 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:55:10,424 INFO [RS:2;jenkins-hbase4:39613] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-19 23:55:10,425 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 329994419c4b206406658a465b604c17 1/1 column families, dataSize=78 B heapSize=488 B 2023-11-19 23:55:10,425 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-11-19 23:55:10,425 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-11-19 23:55:10,425 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1478): Online Regions={329994419c4b206406658a465b604c17=hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17., 1588230740=hbase:meta,,1.1588230740} 2023-11-19 23:55:10,425 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-19 23:55:10,425 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-19 23:55:10,425 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-19 23:55:10,425 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-19 23:55:10,425 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-19 23:55:10,426 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-11-19 23:55:10,426 DEBUG [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1504): Waiting on 1588230740, 329994419c4b206406658a465b604c17 2023-11-19 23:55:10,438 DEBUG [RS:0;jenkins-hbase4:36283] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs 2023-11-19 23:55:10,438 INFO [RS:0;jenkins-hbase4:36283] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36283%2C1700438000319:(num 1700438002071) 2023-11-19 23:55:10,438 DEBUG [RS:0;jenkins-hbase4:36283] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,439 INFO [RS:0;jenkins-hbase4:36283] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,439 INFO [RS:0;jenkins-hbase4:36283] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-11-19 23:55:10,439 INFO [RS:0;jenkins-hbase4:36283] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-19 23:55:10,439 INFO [RS:0;jenkins-hbase4:36283] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-19 23:55:10,439 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-19 23:55:10,439 INFO [RS:0;jenkins-hbase4:36283] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-19 23:55:10,440 INFO [RS:0;jenkins-hbase4:36283] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36283 2023-11-19 23:55:10,441 DEBUG [RS:1;jenkins-hbase4:34057] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs 2023-11-19 23:55:10,441 INFO [RS:1;jenkins-hbase4:34057] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34057%2C1700438000374:(num 1700438002071) 2023-11-19 23:55:10,441 DEBUG [RS:1;jenkins-hbase4:34057] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,441 INFO [RS:1;jenkins-hbase4:34057] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,442 INFO [RS:1;jenkins-hbase4:34057] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-11-19 23:55:10,443 INFO [RS:1;jenkins-hbase4:34057] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-19 23:55:10,443 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-19 23:55:10,443 INFO [RS:1;jenkins-hbase4:34057] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-19 23:55:10,443 INFO [RS:1;jenkins-hbase4:34057] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-19 23:55:10,444 INFO [RS:1;jenkins-hbase4:34057] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34057 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34057,1700438000374 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:55:10,450 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:55:10,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,451 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36283,1700438000319 2023-11-19 23:55:10,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/.tmp/info/c0d0a0b8ee1c4610be14906931edba23 2023-11-19 23:55:10,452 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34057,1700438000374] 2023-11-19 23:55:10,452 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34057,1700438000374; numProcessing=1 2023-11-19 23:55:10,454 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34057,1700438000374 already deleted, retry=false 2023-11-19 23:55:10,454 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,34057,1700438000374 expired; onlineServers=2 2023-11-19 23:55:10,454 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36283,1700438000319] 2023-11-19 23:55:10,454 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36283,1700438000319; numProcessing=2 2023-11-19 23:55:10,455 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36283,1700438000319 already deleted, retry=false 2023-11-19 23:55:10,455 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,36283,1700438000319 expired; onlineServers=1 2023-11-19 23:55:10,455 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/info/c62c426c296a47b58d2d502709f3a02d 2023-11-19 23:55:10,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/.tmp/info/c0d0a0b8ee1c4610be14906931edba23 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/info/c0d0a0b8ee1c4610be14906931edba23 2023-11-19 23:55:10,458 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c62c426c296a47b58d2d502709f3a02d 2023-11-19 23:55:10,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/info/c0d0a0b8ee1c4610be14906931edba23, entries=2, sequenceid=6, filesize=4.8 K 2023-11-19 23:55:10,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 329994419c4b206406658a465b604c17 in 36ms, sequenceid=6, compaction requested=false 2023-11-19 23:55:10,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/namespace/329994419c4b206406658a465b604c17/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-11-19 23:55:10,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:10,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:55:10,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 329994419c4b206406658a465b604c17: 2023-11-19 23:55:10,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1700438002448.329994419c4b206406658a465b604c17. 2023-11-19 23:55:10,474 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/rep_barrier/ecc30512c8204c22a757f241aeab5404 2023-11-19 23:55:10,477 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecc30512c8204c22a757f241aeab5404 2023-11-19 23:55:10,486 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/table/9b81a52a9f28419b90d96ee248f22e60 2023-11-19 23:55:10,490 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b81a52a9f28419b90d96ee248f22e60 2023-11-19 23:55:10,490 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/info/c62c426c296a47b58d2d502709f3a02d as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/info/c62c426c296a47b58d2d502709f3a02d 2023-11-19 23:55:10,493 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c62c426c296a47b58d2d502709f3a02d 2023-11-19 23:55:10,493 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/info/c62c426c296a47b58d2d502709f3a02d, entries=59, sequenceid=210, filesize=11.1 K 2023-11-19 23:55:10,494 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/rep_barrier/ecc30512c8204c22a757f241aeab5404 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/rep_barrier/ecc30512c8204c22a757f241aeab5404 2023-11-19 23:55:10,497 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ecc30512c8204c22a757f241aeab5404 2023-11-19 23:55:10,497 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/rep_barrier/ecc30512c8204c22a757f241aeab5404, entries=24, sequenceid=210, filesize=7.5 K 2023-11-19 23:55:10,497 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/.tmp/table/9b81a52a9f28419b90d96ee248f22e60 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/table/9b81a52a9f28419b90d96ee248f22e60 2023-11-19 23:55:10,500 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b81a52a9f28419b90d96ee248f22e60 2023-11-19 23:55:10,501 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/table/9b81a52a9f28419b90d96ee248f22e60, entries=36, sequenceid=210, filesize=8.0 K 2023-11-19 23:55:10,501 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 76ms, sequenceid=210, compaction requested=false 2023-11-19 23:55:10,506 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-11-19 23:55:10,506 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-11-19 23:55:10,506 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-19 23:55:10,507 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-19 23:55:10,507 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-19 23:55:10,507 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-11-19 23:55:10,508 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,508 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,508 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,553 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,553 INFO [RS:1;jenkins-hbase4:34057] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34057,1700438000374; zookeeper connection closed. 2023-11-19 23:55:10,553 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:34057-0x10008109e390002, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,553 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,553 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@35fcc0d6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@35fcc0d6 2023-11-19 23:55:10,553 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:36283-0x10008109e390001, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,553 INFO [RS:0;jenkins-hbase4:36283] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36283,1700438000319; zookeeper connection closed. 2023-11-19 23:55:10,554 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@564ecf1c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@564ecf1c 2023-11-19 23:55:10,626 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39613,1700438000411; all regions closed. 2023-11-19 23:55:10,631 DEBUG [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs 2023-11-19 23:55:10,631 INFO [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39613%2C1700438000411.meta:.meta(num 1700438002186) 2023-11-19 23:55:10,635 DEBUG [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/oldWALs 2023-11-19 23:55:10,635 INFO [RS:2;jenkins-hbase4:39613] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39613%2C1700438000411:(num 1700438002070) 2023-11-19 23:55:10,635 DEBUG [RS:2;jenkins-hbase4:39613] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,635 INFO [RS:2;jenkins-hbase4:39613] regionserver.LeaseManager(133): Closed leases 2023-11-19 23:55:10,635 INFO [RS:2;jenkins-hbase4:39613] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-11-19 23:55:10,635 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-19 23:55:10,636 INFO [RS:2;jenkins-hbase4:39613] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39613 2023-11-19 23:55:10,639 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39613,1700438000411 2023-11-19 23:55:10,639 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-19 23:55:10,640 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39613,1700438000411] 2023-11-19 23:55:10,640 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39613,1700438000411; numProcessing=3 2023-11-19 23:55:10,641 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39613,1700438000411 already deleted, retry=false 2023-11-19 23:55:10,641 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,39613,1700438000411 expired; onlineServers=0 2023-11-19 23:55:10,641 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45383,1700437998977' ***** 2023-11-19 23:55:10,641 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-11-19 23:55:10,642 DEBUG [M:0;jenkins-hbase4:45383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76855def, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-19 23:55:10,642 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45383,1700437998977 2023-11-19 23:55:10,642 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45383,1700437998977; all regions closed. 2023-11-19 23:55:10,642 DEBUG [M:0;jenkins-hbase4:45383] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-19 23:55:10,642 DEBUG [M:0;jenkins-hbase4:45383] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-11-19 23:55:10,642 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-11-19 23:55:10,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1700438001669] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1700438001669,5,FailOnTimeoutGroup] 2023-11-19 23:55:10,642 DEBUG [M:0;jenkins-hbase4:45383] cleaner.HFileCleaner(317): Stopping file delete threads 2023-11-19 23:55:10,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1700438001670] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1700438001670,5,FailOnTimeoutGroup] 2023-11-19 23:55:10,642 INFO [M:0;jenkins-hbase4:45383] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-11-19 23:55:10,642 INFO [M:0;jenkins-hbase4:45383] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-11-19 23:55:10,643 INFO [M:0;jenkins-hbase4:45383] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-11-19 23:55:10,643 DEBUG [M:0;jenkins-hbase4:45383] master.HMaster(1512): Stopping service threads 2023-11-19 23:55:10,643 INFO [M:0;jenkins-hbase4:45383] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-11-19 23:55:10,643 ERROR [M:0;jenkins-hbase4:45383] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-11-19 23:55:10,643 INFO [M:0;jenkins-hbase4:45383] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-19 23:55:10,643 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-11-19 23:55:10,643 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-11-19 23:55:10,643 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-19 23:55:10,643 DEBUG [M:0;jenkins-hbase4:45383] zookeeper.ZKUtil(398): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-11-19 23:55:10,643 WARN [M:0;jenkins-hbase4:45383] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-11-19 23:55:10,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-19 23:55:10,644 INFO [M:0;jenkins-hbase4:45383] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-19 23:55:10,644 INFO [M:0;jenkins-hbase4:45383] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-11-19 23:55:10,644 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-19 23:55:10,644 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:55:10,644 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:55:10,644 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-19 23:55:10,644 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:55:10,644 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.15 KB heapSize=719.08 KB 2023-11-19 23:55:10,655 INFO [M:0;jenkins-hbase4:45383] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.15 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/aaa6d60c3be644d28425d889747ef270 2023-11-19 23:55:10,658 INFO [M:0;jenkins-hbase4:45383] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aaa6d60c3be644d28425d889747ef270 2023-11-19 23:55:10,659 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/aaa6d60c3be644d28425d889747ef270 as hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/aaa6d60c3be644d28425d889747ef270 2023-11-19 23:55:10,661 INFO [M:0;jenkins-hbase4:45383] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aaa6d60c3be644d28425d889747ef270 2023-11-19 23:55:10,661 INFO [M:0;jenkins-hbase4:45383] regionserver.HStore(1080): Added hdfs://localhost:45125/user/jenkins/test-data/ed6428a2-8fd7-ebb3-3c09-68e0feda66e6/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/aaa6d60c3be644d28425d889747ef270, entries=156, sequenceid=1282, filesize=34.9 K 2023-11-19 23:55:10,662 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegion(2948): Finished flush of dataSize ~608.15 KB/622747, heapSize ~719.06 KB/736320, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 18ms, sequenceid=1282, compaction requested=false 2023-11-19 23:55:10,663 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-19 23:55:10,663 DEBUG [M:0;jenkins-hbase4:45383] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-19 23:55:10,666 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-19 23:55:10,666 INFO [M:0;jenkins-hbase4:45383] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-11-19 23:55:10,667 INFO [M:0;jenkins-hbase4:45383] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45383 2023-11-19 23:55:10,669 DEBUG [M:0;jenkins-hbase4:45383] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,45383,1700437998977 already deleted, retry=false 2023-11-19 23:55:10,729 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-19 23:55:10,740 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,740 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): regionserver:39613-0x10008109e390003, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,740 INFO [RS:2;jenkins-hbase4:39613] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39613,1700438000411; zookeeper connection closed. 2023-11-19 23:55:10,741 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7f7affea] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7f7affea 2023-11-19 23:55:10,741 INFO [Listener at localhost/42385] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-11-19 23:55:10,771 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,771 INFO [M:0;jenkins-hbase4:45383] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45383,1700437998977; zookeeper connection closed. 2023-11-19 23:55:10,771 DEBUG [Listener at localhost/42385-EventThread] zookeeper.ZKWatcher(600): master:45383-0x10008109e390000, quorum=127.0.0.1:52135, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-19 23:55:10,772 WARN [Listener at localhost/42385] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-19 23:55:10,776 INFO [Listener at localhost/42385] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-19 23:55:10,884 WARN [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-11-19 23:55:10,884 WARN [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid 3d39986b-89f7-4439-b67d-be51253160b9) service to localhost/127.0.0.1:45125 2023-11-19 23:55:10,886 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data5/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:10,886 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data6/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:10,889 WARN [Listener at localhost/42385] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-19 23:55:10,891 INFO [Listener at localhost/42385] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-19 23:55:10,995 WARN [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid 92965f72-b1d2-4a0b-81fa-7e4974137a5a) service to localhost/127.0.0.1:45125 2023-11-19 23:55:10,997 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data3/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:10,997 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data4/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:10,999 WARN [Listener at localhost/42385] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-19 23:55:11,001 INFO [Listener at localhost/42385] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-19 23:55:11,102 WARN [BP-950203079-172.31.14.131-1700437995196 heartbeating to localhost/127.0.0.1:45125] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-950203079-172.31.14.131-1700437995196 (Datanode Uuid c1e6d43a-1c05-42be-a588-44ff607a74aa) service to localhost/127.0.0.1:45125 2023-11-19 23:55:11,103 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data1/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:11,103 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/554fc041-dd34-54c5-2e3a-78cb0348438b/cluster_18c34b6b-e2dd-33e4-b469-470501ee891d/dfs/data/data2/current/BP-950203079-172.31.14.131-1700437995196] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-19 23:55:11,128 INFO [Listener at localhost/42385] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-19 23:55:11,319 INFO [Listener at localhost/42385] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-11-19 23:55:11,546 INFO [Listener at localhost/42385] hbase.HBaseTestingUtility(1293): Minicluster is down